# "sum(\[1.0, 1e100, 1.0, -1e100\])" fails

**URL:** <https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502>\
**Category:** General Usage\
**Tags:** question, numbers\
**Created:** [February 15, 2022, 6:37pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502 "2022-02-15T18:37:01Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![runjaj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/runjaj/32/23628_2.png) [@runjaj](https://discourse.julialang.org/u/runjaj)\
**Post date:** [February 15, 2022, 6:37pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/1 "2022-02-15T18:37:01Z")

</div>

I’ve seen a [tweet from David Amos](https://twitter.com/somacdivad/status/1493609641456111616?s=20&t=OyV2aOUpQlUS4kqGzAu_MA) explaining this case where `sum` in Python fails miserably. The example is taken from [BINARY FLOATING POINT SUMMATION ACCURATE TO FULL PRECISION (PYTHON RECIPE)](https://code.activestate.com/recipes/393090/).

Julia fails too:

```julia
julia> sum([1.0, 1e100, 1.0, -1e100])
0.0

```

What would be the easiest way to avoid this error in Julia? In Python, `fsum` gives the correct answer.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [February 15, 2022, 6:43pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/2 "2022-02-15T18:43:43Z")

</div>

```julia
float(sum(Rational.(BigFloat[1.0, 1e100, 1.0, -1e100])))
2.0

```

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [February 15, 2022, 6:45pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/3 "2022-02-15T18:45:26Z")

</div>

```julia
julia> sum(sort([1.0, 1e100, 1.0, -1e100],by=abs,rev=true))
2.0

```

(“easiest” here may mean many things… these are just tricks to get the idea of floating point summation. If that was an actual problem, one would need to check exactly what is the purpose of the calculation to see what to do)

---

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [February 15, 2022, 6:46pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/4 "2022-02-15T18:46:11Z")

</div>

Relevant?

> [@Julia equivalent of Python's "fsum" for floating point summation](https://discourse.julialang.org/t/julia-equivalent-of-pythons-fsum-for-floating-point-summation/17785):
>
> I have been looking for a Julia equivalent of Python’s fsum and have thus far been unable to find one. Is there an implementation already available? If not I will write my own and put it on Github for others to use. From the Python docs: math.fsum ( iterable ) Return an accurate floating point sum of values in the iterable. Avoids loss of precision by tracking multiple intermediate partial sums: The algorithm’s accuracy depends on IEEE-754 arithmetic guarantees and the typical case where the…

---

<div class="post-metadata">

**Author:** ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)\
**Post date:** [February 15, 2022, 6:50pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/5 "2022-02-15T18:50:49Z")

</div>

A third one if the problem are supposed to have “nice” integer numbers, just use `BigInt`s.

```julia
sum([1, BigInt(10)^100, 1, -BigInt(10)^100])

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 15, 2022, 7:16pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/6 "2022-02-15T19:16:33Z")

</div>

> [@rafael.guerra](#):
>
> `Rational`

that’s kind of cheating… might as well just use more precision?

```julia
julia> setprecision(800) do # you can do this globally
           sum(BigFloat[1.0, 1e100, 1.0, -1e100])
       end
2.0

```

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [February 15, 2022, 7:31pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/7 "2022-02-15T19:31:01Z")

</div>

> [@runjaj](#):
>
> `sum([1.0, 1e100, 1.0, -1e100])`

Try

```julia
using KahanSummation

sum_kbn([1.0, 1e100, 1.0, -1e100])

```

which was earlier in `Base` IIUC. Related [thread](https://discourse.julialang.org/t/julia-equivalent-of-pythons-fsum-for-floating-point-summation/17785).

Edit: you’ll also find [AccurateArithmetics](https://github.com/JuliaMath/AccurateArithmetic.jl) and the interesting task to extend error-free transformations.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 15, 2022, 8:54pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/8 "2022-02-15T20:54:17Z")

</div>

> [@runjaj](#):
>
> In Python, `fsum` gives the correct answer.

The direct analogue of `fsum` in Julia is [Xsum.jl](https://github.com/JuliaMath/Xsum.jl), not Kahan/compensated summation (which is very accurate, but is not exactly rounded in general).

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [February 15, 2022, 9:07pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/9 "2022-02-15T21:07:59Z")

</div>

> [@stevengj](#):
>
> The direct analogue of `fsum` in Julia is [Xsum.jl](https://github.com/JuliaMath/Xsum.jl), not Kahan/compensated summation (which is very accurate, but is not exactly rounded in general).

[This](https://github.com/python/cpython/blob/a0ce375e10b50f7606cb86b072fed7d8cd574fe7/Modules/mathmodule.c#L1259-L1417) looks like it is working with a double precision compensated sum?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 15, 2022, 9:11pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/10 "2022-02-15T21:11:29Z")

</div>

In general, this is not a language issue. Python is not “failing”, nor is Julia. You’ll get exactly the same thing if you just use `+` in _any_ language using double-precision floating-point arithmetic (the default precision in most languages).

As another example, `sum([1e-100, 1.0, 1e-100, -1.0])` also gives `0.0` — it is exactly the same problem, just rescaled, in which the exactly rounded answer is `2e-100`. Is this a big error or a small error?

The key question is, _compared to what_.

- It is a big error (100%) compared to the correct sum (i.e. the relative forward error). That’s because we are computing an _ill-conditioned_ function (a sum with a [catastrophic cancellation](https://en.wikipedia.org/wiki/Catastrophic_cancellation)), so the relative error in the output is large even for tiny roundoff errors. You have to be careful when doing _anything_ ill-conditioned in finite precision.

- It is a small error compared to the magnitude of the _inputs_, i.e. the typical scale of the problem we are giving it (`1e100` in your example, `1.0` in mine). A more precise way of saying this is that the “backwards relative error” is small.

Summation is the easiest case to analyze, and the easiest for which you can write exactly rounded algorithms like `fsum` or `xsum` that do the computation in effectively infinite precision, as well as nearly perfect algorithms like compensated summation. But this simplicity is a two-edged sword — it also means that summation perhaps gets disproportionate attention, when in fact ill-conditioning and similar problems are something you have to be aware for _anything_ in finite-precision arithmetic.

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [February 15, 2022, 9:17pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/11 "2022-02-15T21:17:36Z")

</div>

> [@stevengj](#):
>
> In general, this is not a language issue.

Agreed. But this could be about language guarantees like speed vs. accuracy (discussions, which we had before about for example checked integer arithmetic or signalling `NaN`’s).

---

<div class="post-metadata">

**Author:** ![StevenSiew](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevensiew/32/218393_2.png) [@StevenSiew](https://discourse.julialang.org/u/StevenSiew)\
**Post date:** [February 15, 2022, 10:13pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/12 "2022-02-15T22:13:24Z")

</div>

People have mentioned this problem before.

This is **NOT** a problem with the programming language.

This is the **LIMITATION** of the Float64 binary floating point representation of REAL NUMBERS

There is only so much precision in Float64, if you add a small number with a big number, you can only represent the result with so many precision. So you start to lose precision if you exceed the limited precision of Float64.

You will have exactly the same problem with BigFloat too if you exceed its precision as shown below

```julia
julia> sum([big"1.0", big"1e20", big"1.0", big"-1e20"])
2.0

julia> sum([big"1.0", big"1e40", big"1.0", big"-1e40"])
2.0

julia> sum([big"1.0", big"1e80", big"1.0", big"-1e80"])
0.0

```

Even my own decimal floating point module has **LIMITATIONS**

```julia
julia> using DFPs # This is my own personal module
[Info: Precompiling DFPs [top-level]

julia> DFP_setDefaultPrecision(10_000)
10000

julia> DFP_setShowStyle(:short) # don't show 10,000 digits in the results
:short

julia> sum([d"1.0", d"1e80", d"1.0", d"-1e80"])
2.00000

julia> sum([d"1.0", d"1e9999", d"1.0", d"-1e9999"])
2.00000

julia> sum([d"1.0", d"1e10000", d"1.0", d"-1e10000"])
0.00000

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 15, 2022, 10:48pm UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/13 "2022-02-15T22:48:34Z")

</div>

> [@goerch](#):
>
> But this is could be about language guarantees like speed vs. accuracy

No, it could not.

There is no practical way for any language to guarantee exactly rounded results from all possible sequences of floating-point computations. (And there’s a questionable benefit to doing this for only a tiny number of functions like `sum`.)

This is very different from the issue of checking for integer overflow, which might indeed be practical in some cases (or maybe someday in all cases) because it is a purely _local_ check on an _individual_ elementary operation like `+`. In an ill-conditioned computation, in contrast, _every individual step can be correctly rounded_ but the final answer can be completely wrong.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 16, 2022, 12:03am UTC](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/14 "2022-02-16T00:03:49Z")

</div>

> [@StevenSiew](#):
>
> ```julia-auto
> julia> sum([big"1.0", big"1e80", big"1.0", big"-1e80"])
> 0.0
> 
> ```

just use more precision:

> [@"sum(\[1.0, 1e100, 1.0, -1e100\])" fails](https://discourse.julialang.org/t/sum-1-0-1e100-1-0-1e100-fails/76502/6):
>
> that’s kind of cheating… might as well just use more precision? julia\> setprecision(800) do # you can do this globally sum(BigFloat[1.0, 1e100, 1.0, -1e100]) end 2.0

you’re just exceeding the DEFAULT precision