# Missing data and NamedTuple compatibility

**URL:** <https://discourse.julialang.org/t/missing-data-and-namedtuple-compatibility/8136>\
**Category:** Internals & Design\
**Created:** [January 3, 2018, 5:52pm UTC](https://discourse.julialang.org/t/missing-data-and-namedtuple-compatibility/8136 "2018-01-03T17:52:39Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [January 4, 2018, 10:04am UTC](https://discourse.julialang.org/t/missing-data-and-namedtuple-compatibility/8136/6 "2018-01-04T10:04:33Z")

</div>

That’s a serious problem, which has been discussed previously in [this issue](https://github.com/JuliaData/Missings.jl/issues/6). Thanks for bringing it up again and proposing solutions! The `NullableNamedTuples` idea sounds clever. However, I would really like these to work by default with plain `NamedTuple`. Your post prompted me to start a discussion again with some of the core developers, and it looks like we agree on a solution which involves changing how `collect` and `map` compute the element type of the returned array.

The idea is that instead of using the “raw” element type `Union{NamedTuple{{:x, :y}, Tuple{Int64, Int64}}, NamedTuple{{:x, :y}, Tuple{Missing, Int64}}, NamedTuple{{:x, :y}, Tuple{Int64, Missing}}, NamedTuple{{:x, :y}, Tuple{Missing, Missing}}}`, these functions would detect that this type is a `Union` of `NamedTuple` types with different type parameters, and would move the `Union` to the type parameter themselves, giving `NamedTuple{{:x, :y}, Tuple{Union{Int64, Missing}, Union{Int64, Missing}}`. Like your `NullableNamedTuples` approach, a column would only allow for missing values if some missing values are actually present (this is how `map` works for all types so this cannot really be changed).

This is actually related to the currently open [PR 24332](https://github.com/JuliaLang/julia/pull/24332), which is essential to get `map` to work with `missing` (even apart from issues related to tuples). But it appears it would make sense to go even further and use a mechanism almost identical to `promote` to choose the element type. The only change compared with `promote` is that the computed type must always be able to represent all the values exactly, which doesn’t work currently e.g. for `promote(1.0, typemax(Int64))`. So we agreed that we would need a separate mechanism, tentatively called `promote_strict` for that. `promote` could actually automatically fall back on that function, so that most types only need to implement the former.

Help would be welcome to experiment this if somebody is interested.

---

_[View the full topic](https://discourse.julialang.org/t/missing-data-and-namedtuple-compatibility/8136)._