# Sum of float64 vector gives slightly incorrect answer

**URL:** <https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577>\
**Category:** Performance\
**Tags:** question\
**Created:** [January 24, 2018, 10:38pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577 "2018-01-24T22:38:46Z")\
**Posts on this page:** 5\
**Page:** 3

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 29, 2019, 10:41am UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/41 "2019-04-29T10:41:42Z")

</div>

> [@davide](#):
>
> (e.g. H1 - H2, and getting not zero values when I expected zero).

This is normal, use eg `isapprox` with a low tolerance (eg `sqrt(eps())`, `50*eps()`, depending on algorithm, YMMV) for testing.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [April 29, 2019, 12:09pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/42 "2019-04-29T12:09:11Z")

</div>

> [@StefanKarpinski](#):
>
> It would probably be a bit better to separate into positive and negative values and sort each half by ascending absolute value.

This is apparently an old idea, but it turns out not to help in general. It was analyzed in [Higham (1993)](https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.43.3535) for a variety of orderings, who wrote:

> … this “+/–” method does not reduce the amount of cancellation—it simply concentrates all the cancellation in one step. … In summary, the +/– method appears to have no advantages over the other methods considered here, and in cases where there is heavy cancellation in the sum it can be expected to be the least accurate method.

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [April 29, 2019, 2:56pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/43 "2019-04-29T14:56:48Z")

</div>

> [@stevengj](#):
>
> This is apparently an old idea, but it turns out not to help in general

Yes, thanks for pointing this out.

However, it can help when combined with compensation techniques. For example, Priest’s [doubly compensated summation](https://www.researchgate.net/publication/2740764_On_Properties_of_Floating_Point_Arithmetics_Numerical_Stability_and_the_Cost_of_Accurate_Computations) is guaranteed to achieve a 2 ulp accuracy by sorting its operands (in contrast to most other doubly compensated algorithms, which only triple the available precision).

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [April 29, 2019, 3:08pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/44 "2019-04-29T15:08:27Z")

</div>

> [@davide](#):
>
> a = [0.1, 0.1, 100.0];

You may like using [KahanSummation.jl](https://github.com/JuliaMath/KahanSummation.jl)

```julia

julia> using KahanSummation

julia> a = [0.1, 0.1, 100.0];

julia> sum(a), sum(reverse(a))
(100.2, 100.19999999999999)

julia> sum_kbn(a), sum_kbn(reverse(a))
(100.2, 100.2)

```

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [April 29, 2019, 3:19pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/45 "2019-04-29T15:19:07Z")

</div>

I know this has been a long and wide-ranging discussion, but I feel like it’s incomplete without linking to Stefan’s post about the ability to sum a list of 2046 elements to _any floating point value you so choose_ just based upon its ordering. Because sometimes it’s helpful to see the absolute worst case while you’re chasing the best.

> [@Array ordering and naive summation](https://discourse.julialang.org/t/array-ordering-and-naive-summation/1929):
>
> One of the first things you learn in numerical analysis is that floating-point operations are not associative. A classic example is this: julia\> (0.1 + 0.2) + 0.3 0.6000000000000001 julia\> 0.1 + (0.2 + 0.3) 0.6 I was thinking of ways to make floating-point summation independent of the order of the summands without making the performance much worse (this is a hobby of mine). Julia currently uses a pairwise summation algorithm, which is much better than naive left-to-right reduction while havin…

[Previous page](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577.md?page=2)