# Sum of float64 vector gives slightly incorrect answer

**URL:** <https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577>\
**Category:** Performance\
**Tags:** question\
**Created:** [January 24, 2018, 10:38pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577 "2018-01-24T22:38:46Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![GantZA](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gantza/32/4867_2.png) [@GantZA](https://discourse.julialang.org/u/GantZA)\
**Post date:** [January 24, 2018, 10:38pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/1 "2018-01-24T22:38:46Z")

</div>

I’ve noticed that when I run the following code

```julia
a = [0.00694 0.0023 -0.02064 0.01405]
sum(a)

```

but instead of the expected answer of 0.00265 I get 0.0026500000000000013  
Normally this wouldn’t be a problem but this is happening a lot in my program and it changes the results by significant decimal points

---

<div class="post-metadata">

**Author:** ![jebej](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jebej/32/1784_2.png) [@jebej](https://discourse.julialang.org/u/jebej)\
**Post date:** [January 24, 2018, 11:17pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/2 "2018-01-24T23:17:35Z")

</div>

Most decimal numbers cannot be represented exactly by a base 2 floating point number. Please read the following in the manual: [https://docs.julialang.org/en/stable/manual/integers-and-floating-point-numbers/#Machine-epsilon-1](https://docs.julialang.org/en/stable/manual/integers-and-floating-point-numbers/#Machine-epsilon-1)

If you need to add decimal numbers exactly, you will need to represent your numbers differently. Eg with the DecFP package.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 24, 2018, 11:32pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/3 "2018-01-24T23:32:42Z")

</div>

Note also that this happens in _every_ language, since every popular language uses the hardware (binary) floating-point arithmetic by default. You might not notice it in some languages, e.g. Matlab, simply because they print fewer digits by default.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 25, 2018, 1:59am UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/4 "2018-01-25T01:59:10Z")

</div>

> [@stevengj](#):
>
> Note also that this happens in every language

Well, ANSI M/MUMPS (used by a large percentage of the major healthcare applications, especially in the US), uses decimal floating point, so things like a for loop can be done like this: `For i = 0:0.3:0.1 { Write !,i }` (the step is optional, and is last, not in the middle like Julia) would print `"\n0\n0.1\n0.2\n0.3"`.  
No worrying about string ↔ floating point conversions.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [January 25, 2018, 9:46am UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/5 "2018-01-25T09:46:33Z")

</div>

As suggested above, a way to get higher accuracy is to use higher precision arithmetic for intermediate calculations.

For example, you can parse as `BigFloat` the string representations of the numbers, sum the `BigFloat`s, then convert back to `Float64`:

```julia
julia> a = [0.00694 0.0023 -0.02064 0.01405]
1×4 Array{Float64,2}:
 0.00694 0.0023 -0.02064 0.01405

julia> Float64(sum(parse.(BigFloat, string.(a))))
0.00265

```

This is far from being efficient, but that’s the cost for higher accuracy.

---

<div class="post-metadata">

**Author:** ![GantZA](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gantza/32/4867_2.png) [@GantZA](https://discourse.julialang.org/u/GantZA)\
**Post date:** [January 25, 2018, 12:26pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/6 "2018-01-25T12:26:01Z")

</div>

Thanks for the responses. This inaccuracy only occurs every few iterations of my program, and as everything depends on previous iterations, the results after about 1000 iterations are very different from what they used to be when the vector was an Array{Float64, 2}

Maybe if I explain what I’m doing, there might be a simple fix. I have a matrix of asset returns with each row holding one days returns, so I use ` mu = transpose(mapslices(mean, exp_tup, [1]))` to get the column means. This returns an Array{Float64,2}. I only got the floating point inaccuracy when converting the array to a vector using the code ` mu = transpose(mapslices(mean, exp_tup, [1]))[1:end]` which I needed when using the `mu` in the quadprog function. However, I’ve changed how I go about this conversion and now only convert when calling the function like so: `OptimSemiLog(mu[:,1], sigma, risk_aver_vec[L]) `. I haven’t checked if this results in the floating point inaccuracy but I noticed before it was only occuring when the number of rows = 4

OptimSemiLog is my function which just calls the QuadProg function

```julia
function OptimSemiLog(mu, Sigma, risk_aver)
    A = transpose(ones(size(mu)))
    sol = quadprog(mu, Sigma*risk_aver, A, '=', 1, 0, 1, IpoptSolver(print_level=0))
    return sol.sol
end 

```

If I just left mu, I got a type error.

A further question, how would I go about detecting where these floating point inaccuracies occur? I only picked up on this happening by chance

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 25, 2018, 12:41pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/7 "2018-01-25T12:41:31Z")

</div>

Although it is in a package, [https://github.com/stevengj/DecFP.jl](https://github.com/stevengj/DecFP.jl), it will give you the results you expect, and is a good deal faster than `BigFloat` also. (It seemed about 3x faster overall, when I was benchmarking it a couple of years ago, and memory usage is much better).

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 25, 2018, 12:49pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/8 "2018-01-25T12:49:59Z")

</div>

> [@GantZA](#):
>
> A further question, how would I go about detecting where these floating point inaccuracies occur?

Basically everywhere you use floating point, unless both the arguments and the result are exactly representable, which happens mostly by chance. It is best to just assume that this _will_ happen, then deal with it in some appropriate manner. In this case, you could

1. just print fewer digits (if it bothers the user),
2. use `isapprox` to compare results,
3. round, renormalize, etc.

It all depends on your problem.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 25, 2018, 1:10pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/9 "2018-01-25T13:10:21Z")

</div>

> [@ScottPJones](#):
>
> Although it is in a package, [GitHub - JuliaMath/DecFP.jl: Julia IEEE decimal floating-point via the Intel decimal-float library](https://github.com/stevengj/DecFP.jl), it will give you the results you expect

Note that decimal floating-point arithmetic only eliminates roundoff errors for converting human decimal inputs into the computer’s representations. Arithmetic operations can still easily incur roundoff errors if the result is not exactly represented in decimal with the current precision.

The poster is apparently calling a quadratic-programming solver, which will almost certainly incur roundoff errors in both binary and decimal floating-point arithmetic. Also, for calling Ipopt you’ll need binary floating-point values anyway.

The key point here is that roundoff errors are inherent to floating-point arithmetic. This will happen in general regardless of whether binary or decimal is used, and regardless of the precision (`Float64` or `Dec128` or `BigFloat`). You need to get used to this and comfortable with its properties if you are going to do floating-point arithmetic (which is typically the only practical way to do large-scale computation) in any language.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 25, 2018, 1:14pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/10 "2018-01-25T13:14:06Z")

</div>

> [@stevengj](#):
>
> Also, for calling Ipopt you’ll need binary floating-point values anyway.

Some math stuff I wasn’t aware of!

Does that function work with `ArbFloat`?  
I think those are faster than `BigFloat`, and can be more accurate?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 25, 2018, 1:20pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/11 "2018-01-25T13:20:21Z")

</div>

> [@ScottPJones](#):
>
> Also, for calling Ipopt you’ll need binary floating-point values anyway.

Just to be clear, I was referring to the fact that Ipopt is an external C++ library, and like most numerical C/C++ libraries (even in C++) I’m pretty sure it’s hard-coded only to use the hardware floating-point types (and is compiled in double precision by default). You can’t pass it any other floating-point type.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 25, 2018, 1:26pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/12 "2018-01-25T13:26:18Z")

</div>

> [@stevengj](#):
>
> Ipopt is an external C++ library, and like most numerical C/C++ libraries (even in C++)

Precisely why we need more of those libraries rewritten in nice generic Julia!  
Thanks for the clarifications!

---

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [January 25, 2018, 2:32pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/13 "2018-01-25T14:32:49Z")

</div>

Why isn’t DecFP ( IEEE 754-2008 ) used as the base/regular way to represent numbers in Julia?

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 25, 2018, 2:40pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/14 "2018-01-25T14:40:21Z")

</div>

I’m probably one of the people who most wanted/needed decimal floating point support (for databases) when I first started using Julia (and at-stevengj almost instantly wrapped the Intel library), but since most of the Julia users are looking for scientific/numerical/technical computing, I don’t think that would make sense.  
(Having DecFP.jl in stdlib, but not loaded by default, I think might be a good idea, though)

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [January 25, 2018, 2:43pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/15 "2018-01-25T14:43:39Z")

</div>

Because TwoFP is the common case? With floating point in general you create a nonlinear system and DecFP only _looks_ like more natural.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 25, 2018, 2:53pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/16 "2018-01-25T14:53:22Z")

</div>

> [@Juan](#):
>
> Why isn’t DecFP ( IEEE 754-2008 ) used as the base/regular way to represent numbers in Julia?

The nice thing about Julia is that it supports a wide variety of numeric types, `ArbFloat`s, `BigFloat`s, IEEE floats (both binary and decimal), `Unums`, and allows you to use them with other generic numeric types, such as `Complex`.  
What other language allows you all that? 😀

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 25, 2018, 4:27pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/17 "2018-01-25T16:27:19Z")

</div>

> [@Juan](#):
>
> Why isn’t DecFP ( IEEE 754-2008 ) used as the base/regular way to represent numbers in Julia?

Because decimal floating-point is implemented only in software (on most CPUs), whereas binary floating-point is implemented in hardware and therefore inevitably much faster, no matter how cleverly you implement the decimal arithmetic.

And again, decimal floating point only eliminates _one_ kind of roundoff error: roundoff in converting human (decimal) inputs into the computer’s type. This is extremely useful in some applications, e.g. finance! But the vast majority of technical computing performs additional calculations on the inputs, where you will almost inevitably incur roundoff errors even in decimal floating-point. For things like scientific modeling, statistics, and machine learning, there is essentially no benefit to decimal floating-point, and so performance considerations dominate.

It is quite easy to use decimal floating-point in your Julia code if you want it (the [ChangePrecision package](https://github.com/stevengj/ChangePrecision.jl) makes it even easier, for simple scripts), and for some applications it is crucial. But it is a mistake to view it as a panacea. If you do computing with non-integer values, you _need_ to learn something about roundoff errors and computer arithmetic.

---

<div class="post-metadata">

**Author:** ![John\_Gibson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/john_gibson/32/5321_2.png) [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)\
**Post date:** [January 25, 2018, 4:50pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/18 "2018-01-25T16:50:12Z")

</div>

Seems like it’s time for a “PSA: floating-point arithmetic”, like Thomas\_Papp’s “PSA: How to quote code with backticks”. I’ll write one unless someone beats me to it.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 25, 2018, 5:17pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/19 "2018-01-25T17:17:00Z")

</div>

> [@stevengj](#):
>
> It is quite easy to use decimal floating-point in your Julia code if you want it (the ChangePrecision package makes it even easier, for simple scripts), and for some applications it is crucial.

Does that work for decimal floating-point correctly? I remember a problem I had when I started with Julia, I found out that the femto-lisp parser converts floating point literals to `Float64`, and doesn’t keep the original string. It would be good if `Meta.parse` (possibly only with a keyword argument) would wrap literals (at least those that were not “canonic”) with an `Expr` that has the converted value along with the original string parsed.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 25, 2018, 5:58pm UTC](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577/20 "2018-01-25T17:58:36Z")

</div>

> [@ScottPJones](#):
>
> Does that work for decimal floating-point correctly? I remember a problem I had when I started with Julia, I found out that the femto-lisp parser converts floating point literals to `Float64`, and doesn’t keep the original string.

Mostly yes.

ChangePrecision works by taking float literals (which are indeed stored in the AST as `Float64`) and printing them to a string, then re-parsing this string in the desired numeric type. In almost all cases, the grisu algorithm used to print floating-point values will print something equivalent to the original decimal input string, since it prints the shortest string that parses to the same value.

For example, `0.3` parses to a `Float64` value that is not quite `0.3`, but it _prints_ as `"0.3"`, so ChangePrecision will preserve this exact value when converting to `Dec64`, or for `BigFloat` it will give you `BigFloat("0.3")`. However, it is not 100% reliable — in the unlikely event that you enter the literal `0.099999999999999999`, ChangePrecision will treat it as if you had typed `"0.1"` (`== repr(0.099999999999999999)`).

ChangePrecision is convenient for quick hacks, I think — taking an existing script and quickly experimenting with another precision. But in the long run, you should strive to write libraries of code that work for any precision (indeed, any `Number` type), and if you need absolute guarantees of representing human inputs exactly you should probably use DecFP (or similar) directly.

[Next page](https://discourse.julialang.org/t/sum-of-float64-vector-gives-slightly-incorrect-answer/8577.md?page=2)