# Statistics.var wasting time calculating the mean?

**URL:** <https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944>\
**Category:** General Usage\
**Tags:** statistics\
**Created:** [June 7, 2020, 7:37pm UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944 "2020-06-07T19:37:49Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [June 7, 2020, 7:37pm UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944/1 "2020-06-07T19:37:49Z")

</div>

I was trying to understand why Statistics.jl has both `var` and `varm` functions (since `var` already accepts a `mean` parameter). Doing `@less var([1,2,3])` I see:

```julia
var(A::AbstractArray; corrected::Bool=true, mean=nothing, dims=:) = _var(A, corrected, mean, dims)

_var(A::AbstractArray, corrected::Bool, mean, dims) =
    varm(A, something(mean, Statistics.mean(A, dims=dims)); corrected=corrected, dims=dims)

_var(A::AbstractArray, corrected::Bool, mean, ::Colon) =
    real(varm(A, something(mean, Statistics.mean(A)); corrected=corrected))

```

Am I missing something here, or is the method recalculating the mean even when it is provided? Isn’t it wasteful?

Bonus question: I’m still not clear why Statistics exposes both `var` and `varm`…

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [June 7, 2020, 8:19pm UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944/2 "2020-06-07T20:19:51Z")

</div>

It looks like the default value for mean in the first method is nothing. It is eventually passed to

`varm(A, something(mean, Statistics.mean(A, dims=dims)); corrected=corrected, dims=dims)`

If the the first argument in `something` is `nothing`, the result is

```julia
julia> something(nothing, 2)
2

```

if the first argument in `something` is not nothing, the result is:

```julia
julia> something(3, 2)
3

```

So it appears that the mean is not recalculated if it is provided.

I’m not sure why there are two methods with the same functionality. My guess is that `varm` might be deprecated at some point, or might eventually be for internal use only. Good question.

---

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [June 7, 2020, 8:27pm UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944/3 "2020-06-07T20:27:34Z")

</div>

I believe these functions existed before Julia had keyword arguments.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 7, 2020, 8:51pm UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944/4 "2020-06-07T20:51:24Z")

</div>

> [@Christopher\_Fisher](#):
>
> If the the first argument in `something` is `nothing` , the result is
> 
> ```julia
> julia> something(nothing, 2)
> 2
> 
> ```
> 
> if the first argument in `something` is not nothing, the result is:
> 
> ```julia
> julia> something(3, 2)
> 3
> 
> ```
> 
> So it appears that the mean is not recalculated if it is provided.

Since `something` is a function and not control flow, I think it must evaluate both arguments, even if it then chooses to return the first. So to me it looks like it still recalculates the mean, which is surprising.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [June 7, 2020, 9:04pm UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944/5 "2020-06-07T21:04:06Z")

</div>

This is quite easy to test for yourself:

```julia
julia> using Statistics, BenchmarkTools

julia> let v = randn(1_000)
           @btime var($v)
           @btime var($v, mean=$(mean(v)))
           @btime varm($v, $(mean(v)))
           @btime mean($v)
       end;
  197.689 ns (0 allocations: 0 bytes)
  195.053 ns (0 allocations: 0 bytes)
  117.394 ns (0 allocations: 0 bytes)
  79.235 ns (0 allocations: 0 bytes)

```

Yes, it’s quite clear that `var(v)` and `var(v; mean=meanv)` have essentially identical timings, whereas the difference between their times and the time for `varm(v, meanv)` is essentially exactly the time it takes to calculate `mean(v)`.

I’m sure a PR to Statistics.jl rectifying this would be quite welcome.

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [June 7, 2020, 10:52pm UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944/6 "2020-06-07T22:52:46Z")

</div>

I assumed that `something` contained control flow logic to prevent unnecessary evaluations. Good catch!

Edit: Oh man. I just realized that control flow doesn’t even matter because the mean has to be calculated before it’s passed to `something`. 😆 Perhaps that’s why it was lurking there for years.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [June 8, 2020, 9:35am UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944/7 "2020-06-08T09:35:42Z")

</div>

> [@johnmyleswhite](#):
>
> I believe these functions existed before Julia had keyword arguments.

That makes sense. And there’s already an issue about this: [https://github.com/JuliaLang/Statistics.jl/issues/5](https://github.com/JuliaLang/Statistics.jl/issues/5).

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [June 8, 2020, 9:37am UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944/8 "2020-06-08T09:37:32Z")

</div>

@Mason thanks for the benchmarks!

> [@Mason](#):
>
> I’m sure a PR to Statistics.jl rectifying this would be quite welcome.

I’ll make a PR (in a few days).

---

<div class="post-metadata">

**Author:** ![halleysfifthinc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/halleysfifthinc/32/206280_2.png) [@halleysfifthinc](https://discourse.julialang.org/u/halleysfifthinc)\
**Post date:** [June 8, 2020, 2:22pm UTC](https://discourse.julialang.org/t/statistics-var-wasting-time-calculating-the-mean/40944/9 "2020-06-08T14:22:23Z")

</div>

I believe this is the result of constant prop of `nothing`, which should statically dispatch to the appropriate `something` method.