# How to calculate a weighted mean with missing observations

**URL:** <https://discourse.julialang.org/t/how-to-calculate-a-weighted-mean-with-missing-observations/19281>\
**Category:** Statistics\
**Created:** [January 4, 2019, 8:01pm UTC](https://discourse.julialang.org/t/how-to-calculate-a-weighted-mean-with-missing-observations/19281 "2019-01-04T20:01:56Z")\
**Posts on this page:** 1\
**Showing post:** 12

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [January 5, 2019, 4:16pm UTC](https://discourse.julialang.org/t/how-to-calculate-a-weighted-mean-with-missing-observations/19281/12 "2019-01-05T16:16:56Z")

</div>

From [NA-ignoring aggregations](https://discourse.julialang.org/t/na-ignoring-aggregations/19120), two other considerations regarding NA-like values.

First, does it make sense to make `missing` special? There’s already a [PR open](https://github.com/JuliaLang/julia/pull/30549) that adds `skipnothing` (or `skipoftype`). There may still be cases where it’s preferred to skip `NaN` instead. Should this just be generalized to `skip(somevalue, iterator)`?

Second, consider the case of a matrix with missing values where you’d like to get the means of the rows. Without `missing` values, you’d use `mean(A, dims=1)`. But `mean(skipmissing(A), dims=1)` doesn’t do what you want. Now you can’t really have `SkipMissing` be a shapeful iterator; really `SizeUnknown()` is the only sensible return type for `Base.IteratorSize(::SkipMissing)` even if the underlying iterator has a shape. So I see two options to fix this:

1. Functions like `mean` and `sum` could specifically be overloaded for `Base.SkipMissing{<:AbstractArray}` so that `mean(skipmissing(A), dims=1)` just works.
2. Instead of making `mean(skipmissing(A), dims=1)` work, you could replace it with `[mean(skipmissing(x)) for x in eachslice(A, dims=1)]` (with Julia 1.1).

I think the latter is preferable. I’m not sure how many functions like `mean` and `sum` have a `dims` keyword, but it seems like a pain to make them all be `SkipMissing`-aware. Taking this further, I’m of the opinion that the `dims` arguments for these functions represent a poor separation of concerns (and I know @andyferris shares this opinion, [https://github.com/JuliaArrays/StaticArrays.jl/issues/498#issuecomment-450794143](https://github.com/JuliaArrays/StaticArrays.jl/issues/498#issuecomment-450794143)). So should they just be removed in favor of the `eachslice` version that can be easily made to work with `skipmissing`?

---

_[View the full topic](https://discourse.julialang.org/t/how-to-calculate-a-weighted-mean-with-missing-observations/19281)._