# How to improve runtime with measurements.jl?

**URL:** <https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343>\
**Category:** Performance\
**Tags:** question, measurements\
**Created:** [August 30, 2021, 11:56am UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343 "2021-08-30T11:56:55Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Cevheriferd](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cevheriferd/32/26792_2.png) [@Cevheriferd](https://discourse.julialang.org/u/Cevheriferd)\
**Post date:** [August 30, 2021, 11:56am UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/1 "2021-08-30T11:56:55Z")

</div>

Dear all,

I am using the measurements.jl package to calculate the error in my dataset. I have an array with voltage values which all have their own uncertainty. But when I want to calculate the mean of the array, it takes 27 seconds and allocates 805.25 MiB to do this. The array consists of around 47000 elements all of the same type. Does anyone know how to improve the performance of this operation? For an array of the same size with only Float64 elements the operation is almost done instantly. Thanks for helping me out.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 30, 2021, 12:01pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/2 "2021-08-30T12:01:20Z")

</div>

Welcome! 🙂 It’s hard to say what’s causing allocations or slowing down your code if you don’t share it with us - please share a MWE (minimal working example) showing the problem.

[Please read: make it easier to help you](https://discourse.julialang.org/t/please-read-make-it-easier-to-help-you/14757)

Also, have you followed most of the tips mentioned in the [Performance Tips](https://docs.julialang.org/en/v1/manual/performance-tips/) section of the manual?

---

<div class="post-metadata">

**Author:** ![Cevheriferd](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cevheriferd/32/26792_2.png) [@Cevheriferd](https://discourse.julialang.org/u/Cevheriferd)\
**Post date:** [August 30, 2021, 12:18pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/3 "2021-08-30T12:18:36Z")

</div>

````julia
```julia

````

```julia
using Statistics, Measurements

VoltageArray = rand(50000) #Works really fast

VoltageArray = VoltageArray .± 0.1VoltageArray #Works really fast

mean(VoltageArray) #Takes a long time to run

```

This is a MWE of my problem. I hope I made it clear

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 30, 2021, 1:17pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/4 "2021-08-30T13:17:55Z")

</div>

Divisions are expensive:

```julia
julia> @time (1 ± 0.1) / (2 ± .2)            
  0.000007 seconds (4 allocations: 192 bytes)
0.5 ± 0.071                                  

```

According to [this issue](https://github.com/JuliaPhysics/Measurements.jl/issues/25) on their repo this is known, you may be looking for `weightedmean` instead here:

```julia
julia> v = rand(50_000);                       
                                               
julia> v = v .± 0.1 .* v;                      
                                               
julia> @time weightedmean(v)                   
  0.001619 seconds (6 allocations: 781.438 KiB)
0.0002565 ± 2.2e-6                             

```

Though you should note:

> NOTA BENE: correlation is not taken into account.

Accumulation of error bars is non-trivial, else this would already be much faster, I assume.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 30, 2021, 1:21pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/5 "2021-08-30T13:21:31Z")

</div>

Sure, divisions, but look at this:

```julia
jl> v = rand(100);

jl> vm = v .± 0.1 .* v;

jl> @btime sum($v)
  9.409 ns (0 allocations: 0 bytes)
54.45849425120919

jl> @btime sum($vm)
  14.700 μs (102 allocations: 6.44 KiB)
54.46 ± 0.62

```

That’s a 1500x slowdown(!) Is that really expected?

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 30, 2021, 1:28pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/6 "2021-08-30T13:28:48Z")

</div>

> [@DNF](#):
>
> That’s a 1500x slowdown(!) Is that really expected?

Not my words 🤷‍♂️

> <https://github.com/JuliaPhysics/Measurements.jl/issues/25>
>
> It looks like this package has a lot of overhead for simple math operations - ab…out 50 times slower than normal floats and uses 20 times more memory! Not even talking about number of allocations.
> 
> \`\`\`julia
> using BenchmarkTools
> using Measurements
> 
> a = randn(1000, 1000)
> b = randn(1000, 1000)
> @btime a ./ b;
> \# 1.791 ms (4 allocations: 7.63 MiB)
> a, b = a .± b, b .± a
> @btime a ./ b;
> \# 88.498 ms (4000004 allocations: 175.48 MiB)
> \`\`\`
> 
> For comparison, an extremely basic implementation from \[rosettacode\](https://rosettacode.org/wiki/Numeric\_error\_propagation) has only 4x overhead, which is completely reasonable:
> \`\`\`julia
> using Main.NumericError
> 
> a = randn(1000, 1000)
> b = randn(1000, 1000)
> a, b = Measure.(a, b), Measure.(b, a)
> @btime a ./ b;
> \# 6.797 ms (4 allocations: 15.26 MiB)
> \`\`\`
> 
> Any chance that performance of Measurements.jl can be improved?

Maybe something has changed since then, @giordano could chime in, but I doubt anything grand has changed here. Keeping track of correlations is hard.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [August 30, 2021, 2:13pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/7 "2021-08-30T14:13:14Z")

</div>

Yes, unfortunately that’s exepcted, I made the example of the `mean` in the issue linked above. As @Sukera pointed out, tracking correlation is hard. It’s pretty easy to write a package to propagate uncertainties super quickly ignoring correlations, this is what `Measurements.jl` did until [v0.02](https://github.com/JuliaPhysics/Measurements.jl/releases/tag/v0.0.2), but that’s also incredibly dumb and useless: almost no identies would hold, for example `x + x` and `2 * x` would give you different results.

Regarding the mean in particular, note that most of the time users want to compute the weighted mean, for which `Measurements.jl` provides a [specific function](https://juliaphysics.github.io/Measurements.jl/stable/usage/#Weighted-Average), which should have much more reasonable performance. Note that it ignores correlation, as warned in the docstring, because you’d usually apply to a sample of independent measurements anyways.

To be clear, `Measurements.jl` is slow because of an algorithmic limitation: it uses an O(n^2) algorithm to propagate uncertainties, you can understand why `mean`/`sum` are particularly bad, and get worse and worse as the size of the vector increases. There may be clever ways to reduce the complexity of the algorithm, but I never had the time to look at it. I believe the Python package [`uncertainties`](https://github.com/lebigot/uncertainties/) now has an algorithm which is O(n), or anyways better than O(n^2), but until a few years ago it was using basically the same algorithm as `Measurements.jl` (well, historically it’s the other way around). If anyone is willing to help, I’d be glad to hear from them.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 30, 2021, 2:20pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/8 "2021-08-30T14:20:15Z")

</div>

Okay, I had no idea, and thought is was the same as interval arithmetic.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [August 30, 2021, 2:26pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/9 "2021-08-30T14:26:42Z")

</div>

No, I believe interval arithmetic doesn’t track correlations at all, which is probably fine for what it’s applied to

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 30, 2021, 3:36pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/10 "2021-08-30T15:36:36Z")

</div>

> [@giordano](#):
>
> I believe the Python package [`uncertainties`](https://github.com/lebigot/uncertainties/) now has an algorithm which is O(n),

Bummer, that project has a weird [custom license](https://github.com/lebigot/uncertainties/blob/master/LICENSE.txt) 😓 It _may_ be a BSD license according to git history, but I’m not a copyright lawyer…

---

<div class="post-metadata">

**Author:** ![gbaraldi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gbaraldi/32/22101_2.png) [@gbaraldi](https://discourse.julialang.org/u/gbaraldi)\
**Post date:** [August 30, 2021, 4:06pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/11 "2021-08-30T16:06:23Z")

</div>

I think it is a 3 clause BSD license [The 3-Clause BSD License | Open Source Initiative](https://opensource.org/licenses/BSD-3-Clause). What that means for a port I don’t know, but it’s that one,

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 30, 2021, 4:35pm UTC](https://discourse.julialang.org/t/how-to-improve-runtime-with-measurements-jl/67343/12 "2021-08-30T16:35:19Z")

</div>

Then we may be in luck - that license only places restrictions on redistributions in binary or source form (a port is as far as I know neither), as well as not endorsing our port with their name (i.e. we should be fine if we write something like “port inspired by”…).