Combinators on Nullables

Internals & Design
1

(This is more of a conceptual question, so I am posting in this category, please reassign if inappropriate. Also, I don’t have a formal CS background, so this may have an obvious answer I missed. Thanks for your understanding.)

Ever since learning about Nullables and reading Joe Duffy’s blog post about the Midori Error Model which was mentioned in #7026, I have been trying to structure my code to handle missing values using these concepts.

Frequently I encounter the following situation: I would like to have some operation which is well-defined on actual values, and I would like to lift it to work on Nullables. To make things concrete, suppose the operation maps a single argument to a single result. There seems to be a standard way of doing this with a combinator, which is called and_then in Rust. My problem with implementing this in Julia is that if the value is null, then I cannot call f, so I don’t have a type to use in the Nullable. For example, if I wanted to implement it like

function and_then{T}(f, x::Nullable{T})
    if isnull(x)
        Nullable{return_type(f, Tuple{T})}() # hypothetical
    else
        Nullable(f(get(x)))
    end
end

then I would need to define return_type. The following works, most of the time, but it seems like a hack:

function return_type(f, types)
    rt = Base.return_types(f, types)
    @assert length(rt) == 1 "Could not infer a single return type."
    rt[1]
end

Examples:

julia>  f(x) = x+1
f (generic function with 1 method)

julia>  and_then(f, Nullable(1))
Nullable{Int64}(2)

julia>  and_then(f, Nullable{Int64}())
Nullable{Int64}()

julia>  and_then(f, Nullable{String}()) # but f does not work on strings
Nullable{Any}()

I could of course provide the return type manually. But this becomes unwieldy when it is rather complex.

Is there a way to implement and_then in Julia in some more natural way? Or if not, is this something I should forget about because this is not a good match for the concepts of the language, or are there plans to make it work better?

1 Like
2

The behavior you describe has actually just been implemented as broadcast in Julia 0.6 (see this PR). We usually refer to it as “natural lifting semantics”.

So you can now just write f.(Nullable(1)).

3 Likes
3

Thanks — I looked at the PRs, and realized that this is already documented.

Now suppose I wanted to implement my own lifting semantics for a type like Rust’s Result, which also encodes the error, not merely missingness. What are the standard building blocks from 0.6 that I should use?

4

You could just override broadcast in a similar way to what this PR did for Nullable if you want to support the same syntax. But you could also use a wrapper or meta function, and pass it the function you want as the first argument, like lift(f, args...). This would be more explicit. Depends on your needs.

5

Consider the following (very stylized) function:

function calculate(data::Vector)
    T = Int                       # how to do this non-manually?
    empty = Nullable{Vector{T}}() # return this when failed
    isempty(data) && return empty
    transformed = similar(data, T)
    for (index,elt) in enumerate(data)
        pre_condition(elt) && return empty
        transformed_elt = some_transform(elt)
        post_condition(transformed_elt) && return empty
        transformed[index] = transformed_elt
    end
    Nullable(transformed)
end

which transforms some data, but will consider the data invalid when certain things hold (ie empty data, pre- and post-transformation conditions), for which it returns a Nullable without a value.

I wonder if there is a way to do this nicer, especially so that I would not have to know the type parameter T of the empty Nullable but somehow get it automatically. Usually it is more complex than in the example above, and when I modify some_transform, it could cease being type-stable, unless I update empty accordingly.

Not a problem if this works for 0.6.0 only, I am slowly transitioning anyway.

2 Likes
6

This problem is why we’ve talking through the compiler changes required to replace the internal representation of Nullable{T} with something like Union{Some{T}, Void} since you wouldn’t need to know the problematic parameter T.

2 Likes
7

Can you please link to an issue where I could follow this discussion?

8

https://github.com/JuliaLang/Juleps/pull/21

2 Likes