# Operations on missing values

**URL:** <https://discourse.julialang.org/t/operations-on-missing-values/9785>\
**Category:** General Usage\
**Tags:** question\
**Created:** [March 18, 2018, 10:31am UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785 "2018-03-18T10:31:15Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 18, 2018, 10:31am UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785/1 "2018-03-18T10:31:15Z")

</div>

In Julia 0.7 what is the consensus about the best practice for handling missing arguments in combination with functions that do not accept `missing` values by design?

What I mean is that currently we have for example:

```julia
julia> x= ["A", "b", "AB", missing]
4-element Array{Union{Missing, String},1}:
 "A"
 "b"
 "AB"
 missing

julia> lowercase.(x)
ERROR: MethodError: no method matching lowercase(::Missing)

```

so what you have to do is:

```julia
broadcast(v-> ismissing(v) ? missing : lowercase(v), x)

```

if I remember there were earlier discussions about this but I could not find the conclusion. What essentially I am looking for is a function similar to `skipmissing` but passing them through like (probably a shorter name would be nice as this function would be heavily used):

```julia
passmissing(f, x) = ismissing(x) ? missing : f(x) # maybe this special case is not needed if the later is fast for f accepting a single argument
passmissing(f, x...) = any(ismissing, x) ? missing : f(x...)

```

and now you can write `passmissing.(lowercase, x)` to get the desired result.

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [March 19, 2018, 3:46pm UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785/2 "2018-03-19T15:46:15Z")

</div>

With `Missing` defined in Base there really should be a generic fallback I guess.

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [March 19, 2018, 4:57pm UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785/3 "2018-03-19T16:57:52Z")

</div>

Maybe `Missings` could provide some `lift(f)` that does:

lift(f)(::Missing) = missing  
lift(f)(x) = f(x)

and maybe some `@lift` macro that would `lift` all functions in a code block, though the details on the macro side may be a bit tricky to figure out exactly.

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [March 19, 2018, 8:35pm UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785/4 "2018-03-19T20:35:44Z")

</div>

Yeah, that’s another possibility, so something like:

```julia
lift(f) = (x...) -> any(ismissing, x) ? missing : f(x...)

```

Though `lift(f)(x)` doesn’t look great compared with `lift(f, x)`. Same for `lift(f).(X)` vs. `lift.(f, X)`.

A `@lift` macro would be nice, at least it would allow keeping normal syntax after it. It shouldn’t be too hard to support as you just need to find `call` nodes and adapt them.

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [March 19, 2018, 10:05pm UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785/5 "2018-03-19T22:05:52Z")

</div>

This may be a very technical point, but I was thinking that one could use a generated function (rather than anonymous function) so that you could check if any of the arguments is missing at compile time, not sure how much it matters. Maybe it’s good in some cases (say few arguments) and bad in others.

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [March 19, 2018, 10:10pm UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785/6 "2018-03-19T22:10:14Z")

</div>

Yes, we would have to be very careful about performance when writing the actual implementation. Thanks to the `any` recursive definition for small tuples, the simple definition could be quite fast, but maybe splatting is too costly and we’d need either a generated function or special cases for small numbers of arguments.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 19, 2018, 11:16pm UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785/7 "2018-03-19T23:16:20Z")

</div>

1. My thinking was that specialized cases for small tuples might be needed but this requires benchmarking (a PR for a similar case is open here [https://github.com/JuliaLang/julia/pull/16604](https://github.com/JuliaLang/julia/pull/16604) with different options compared).
2. I was also considering a macro; then we have to take into account that some functions have handle `missing` argument and this should not be overridden.
3. I prefer `lift(f,x)` syntax as it is shorter and allows `do` syntax. What advantages of `lift(f)(x)` over `lift(f,x)` do you see?

and I guess everyone accepts `lift` as a name, which is great as it is short which is important in this case I think.

---

<div class="post-metadata">

**Author:** ![bramtayl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bramtayl/32/3614_2.png) [@bramtayl](https://discourse.julialang.org/u/bramtayl)\
**Post date:** [March 19, 2018, 11:22pm UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785/8 "2018-03-19T23:22:53Z")

</div>

I thought the whole point of missing was that it was supposed to propagate automatically? Shouldn’t we just get `lowercase(::Missing) = missing`? Lift might be good for `lowercase(::Nothing)`

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 19, 2018, 11:28pm UTC](https://discourse.julialang.org/t/operations-on-missing-values/9785/9 "2018-03-19T23:28:47Z")

</div>

At least for now we would have to define this behavior for all functions that should support it (and the consensus is that for functions operating on strings a PR adding such support should be done).

However, as I have noted above - in general not all should be lifted automatically as some functions might want to accept `missing` as a valid argument. Consider for example `vcat(missing)` it returns 1-element array containing `missing` (as expected) and not `missing`. The same might be with user defined functions.

But I agree that unfortunately this is a bit awkward.