# Statistics.mean() function with a Matrix containing missing values

**URL:** <https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115>\
**Category:** New to Julia\
**Created:** [February 6, 2023, 3:08am UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115 "2023-02-06T03:08:54Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![austin-putz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/austin-putz/32/2583_2.png) [@austin-putz](https://discourse.julialang.org/u/austin-putz)\
**Post date:** [February 6, 2023, 3:08am UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115/1 "2023-02-06T03:08:54Z")

</div>

I have a matrix with missing values and I want to calculate the column means. I’m not sure how to drop missing.

```julia
using Statistics

vec = [1, missing, 2]
mean(vec) # is missing
mean(skipmissing(vec) # give 1.5 that I want in 2 dim

# now I have a Matrix A with missing
A = [1 5
       6 missing]

# unsure how to calculate column means
Statistics.mean(A, dims=1) # error for missing value
Statistics.mean(skipmissing(A), dims=1) # error

```

Everywhere I look I can only find vector examples… I don’t see any options in the mean function to drop missing in 2d. Any help would be appreciated.

---

<div class="post-metadata">

**Author:** ![Krastanov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krastanov/32/6817_2.png) [@Krastanov](https://discourse.julialang.org/u/Krastanov)\
**Post date:** [February 6, 2023, 3:28am UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115/2 "2023-02-06T03:28:00Z")

</div>

`skipmissing` returns a linear iterator (one single axis) without allocating a copy:

```julia
julia> skipmissing(A) |> collect |> size
(3,)

```

The `dims` keyword does not work in that case because we have a single axis.

You can make an iterator that goes over each slice you care about (with `eachslice` or `eachcol` or `eachrow`) and then broadcast `skipmissing` and `mean` on the resulting iterator of single-axis arrays.

```julia
mean.(skipmissing.(eachrow(A)))

```

There might be a cleaner way to do it. The underlying issue is that skipmissing can not return a multi-axis array because different rows might need a different number of skips due to a different number of `missing`s.

---

<div class="post-metadata">

**Author:** ![briochemc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/briochemc/32/4209_2.png) [@briochemc](https://discourse.julialang.org/u/briochemc)\
**Post date:** [February 6, 2023, 4:04am UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115/3 "2023-02-06T04:04:58Z")

</div>

Related issue: [sum and mean of skipmissings don't accept the dims kwarg · Issue #40081 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/40081)

---

<div class="post-metadata">

**Author:** ![austin-putz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/austin-putz/32/2583_2.png) [@austin-putz](https://discourse.julialang.org/u/austin-putz)\
**Post date:** [February 6, 2023, 4:58pm UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115/4 "2023-02-06T16:58:12Z")

</div>

Perfect this line worked:

```julia
mean.(skipmissing.(eachrow(A')))

```

(small edit: I needed column means so I do the transpose of my matrix of number of individuals by number (3k) of SNPs (45k))  
Okay this was my worry, I hope the Statistics package improves this soon to deal with missing as this is more work than it needs to be (imo…). Thank you for your help!

---

<div class="post-metadata">

**Author:** ![austin-putz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/austin-putz/32/2583_2.png) [@austin-putz](https://discourse.julialang.org/u/austin-putz)\
**Post date:** [February 6, 2023, 4:58pm UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115/5 "2023-02-06T16:58:35Z")

</div>

Thank you very much for you suggestion here, I will read this over now.

---

<div class="post-metadata">

**Author:** ![Krastanov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krastanov/32/6817_2.png) [@Krastanov](https://discourse.julialang.org/u/Krastanov)\
**Post date:** [February 6, 2023, 5:54pm UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115/6 "2023-02-06T17:54:07Z")

</div>

There is also `eachcol` which gives an iterator over columns. It does not really matter whether you transpose or whether you switch from `eachrow` to `eachcol`.

---

<div class="post-metadata">

**Author:** ![austin-putz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/austin-putz/32/2583_2.png) [@austin-putz](https://discourse.julialang.org/u/austin-putz)\
**Post date:** [February 6, 2023, 6:54pm UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115/7 "2023-02-06T18:54:35Z")

</div>

Oh shoot, I tried `eachcolumn()` and didn’t work. Thanks I’ll use `eachcol()`

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [February 6, 2023, 9:03pm UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115/8 "2023-02-06T21:03:11Z")

</div>

> [@austin-putz](#):
>
> Okay this was my worry, I hope the Statistics package improves this soon

Well, you lose some performance in `mean.(skipmissing.(eachcol(A)))` compared to potential `mean(skipmissing(A), dims=1)`, but the former is more general: substitute any aggregation instead of `mean` and it’ll work, without special support by the function.

Anyway, there’s a long-stalled PR linked from the issue above ([Support mapreduce over dimensions with SkipMissing by nalimilan · Pull Request #28027 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/28027)), so you may wish to update/promote it if this feature seems important.

---

<div class="post-metadata">

**Author:** ![austin-putz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/austin-putz/32/2583_2.png) [@austin-putz](https://discourse.julialang.org/u/austin-putz)\
**Post date:** [February 6, 2023, 9:52pm UTC](https://discourse.julialang.org/t/statistics-mean-function-with-a-matrix-containing-missing-values/94115/9 "2023-02-06T21:52:34Z")

</div>

Oh I see… Thank you for this information. Well then they must be aware, I’m not much for development, I’m still trying to learn the basics. Julia is kind of a beast compared to R to learn. Thanks for all your help.