# Combinators on Nullables

(This is more of a conceptual question, so I am posting in this category, please reassign if inappropriate. Also, I don’t have a formal CS background, so this may have an obvious answer I missed. Thanks for your understanding.)

Ever since learning about Nullables and reading Joe Duffy’s blog post about the Midori Error Model which was mentioned in #7026, I have been trying to structure my code to handle missing values using these concepts.

Frequently I encounter the following situation: I would like to have some operation which is well-defined on actual values, and I would like to lift it to work on Nullables. To make things concrete, suppose the operation maps a single argument to a single result. There seems to be a standard way of doing this with a combinator, which is called and_then in Rust. My problem with implementing this in Julia is that if the value is null, then I cannot call `f`, so I don’t have a type to use in the Nullable. For example, if I wanted to implement it like

``````function and_then{T}(f, x::Nullable{T})
if isnull(x)
Nullable{return_type(f, Tuple{T})}() # hypothetical
else
Nullable(f(get(x)))
end
end
``````

then I would need to define `return_type`. The following works, most of the time, but it seems like a hack:

``````function return_type(f, types)
rt = Base.return_types(f, types)
@assert length(rt) == 1 "Could not infer a single return type."
rt[1]
end
``````

Examples:

``````julia>  f(x) = x+1
f (generic function with 1 method)

julia>  and_then(f, Nullable(1))
Nullable{Int64}(2)

julia>  and_then(f, Nullable{Int64}())
Nullable{Int64}()

julia>  and_then(f, Nullable{String}()) # but f does not work on strings
Nullable{Any}()
``````

I could of course provide the return type manually. But this becomes unwieldy when it is rather complex.

Is there a way to implement `and_then` in Julia in some more natural way? Or if not, is this something I should forget about because this is not a good match for the concepts of the language, or are there plans to make it work better?

1 Like

The behavior you describe has actually just been implemented as `broadcast` in Julia 0.6 (see this PR). We usually refer to it as “natural lifting semantics”.

So you can now just write `f.(Nullable(1))`.

3 Likes

Thanks — I looked at the PRs, and realized that this is already documented.

Now suppose I wanted to implement my own lifting semantics for a type like Rust’s `Result`, which also encodes the error, not merely missingness. What are the standard building blocks from 0.6 that I should use?

You could just override `broadcast` in a similar way to what this PR did for `Nullable` if you want to support the same syntax. But you could also use a wrapper or meta function, and pass it the function you want as the first argument, like `lift(f, args...)`. This would be more explicit. Depends on your needs.

Consider the following (very stylized) function:

``````function calculate(data::Vector)
T = Int                       # how to do this non-manually?
empty = Nullable{Vector{T}}() # return this when failed
isempty(data) && return empty
transformed = similar(data, T)
for (index,elt) in enumerate(data)
pre_condition(elt) && return empty
transformed_elt = some_transform(elt)
post_condition(transformed_elt) && return empty
transformed[index] = transformed_elt
end
Nullable(transformed)
end
``````

which transforms some data, but will consider the data invalid when certain things hold (ie empty data, pre- and post-transformation conditions), for which it returns a `Nullable` without a value.

I wonder if there is a way to do this nicer, especially so that I would not have to know the type parameter `T` of the empty `Nullable` but somehow get it automatically. Usually it is more complex than in the example above, and when I modify `some_transform`, it could cease being type-stable, unless I update `empty` accordingly.

Not a problem if this works for `0.6.0` only, I am slowly transitioning anyway.

2 Likes

This problem is why we’ve talking through the compiler changes required to replace the internal representation of `Nullable{T}` with something like `Union{Some{T}, Void}` since you wouldn’t need to know the problematic parameter `T`.

2 Likes