# Does the julia intrinsic sum() apply fastmath by default?

**URL:** <https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815>\
**Category:** New to Julia\
**Tags:** fast-math, sum\
**Created:** [February 17, 2023, 11:51pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815 "2023-02-17T23:51:26Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ender\_L](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ender_l/32/47013_2.png) [@Ender\_L](https://discourse.julialang.org/u/Ender_L)\
**Post date:** [February 17, 2023, 11:51pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/1 "2023-02-17T23:51:26Z")

</div>

I noticed the output of sum() is different from custom loops

```julia
julia> function mysum(A)
           out = 0.0
           for ii in eachindex(A)
               out += A[ii]
           end
           return out
       end
mysum (generic function with 1 method)

julia> A=rand(1000);

julia> sum(A)-mysum(A)
-1.1368683772161603e-13

```

In FORTRAN(gfortran) this only happens when -ffastmath is enabled. Is this by design, or there is something else I missed? If this is expected, what other functions have this behavior?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 17, 2023, 11:57pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/2 "2023-02-17T23:57:49Z")

</div>

Julia’s `sum` uses pairwise summation which is more accurate and usually faster than the regular loop.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [February 18, 2023, 12:02am UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/3 "2023-02-18T00:02:19Z")

</div>

More accurate, yes. But faster? My understanding is that it is ‘almost as fast’.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 18, 2023, 12:03am UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/4 "2023-02-18T00:03:49Z")

</div>

Theoretically it should be able to be faster for arrays that fit in cache since there are better dependency chains.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [February 18, 2023, 12:23am UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/5 "2023-02-18T00:23:32Z")

</div>

How short is that then? When testing, I find this loop to be consistently faster than `sum` for lengths from 10^1 to 10^6:

```julia
function mysum(xs)
    val = zero(eltype(xs))
    @simd for i in eachindex(xs)
        @inbounds val += xs[i]
    end
    return val
end

```

It’s a small difference, but consistent.

Compared to a non-simd loop, yes, that would be expected.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [February 18, 2023, 12:43am UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/6 "2023-02-18T00:43:35Z")

</div>

Note that with `@simd` you’re allowing reordering of the loop, which defeats the premise of Oscar’s statement.

---

<div class="post-metadata">

**Author:** ![Ender\_L](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ender_l/32/47013_2.png) [@Ender\_L](https://discourse.julialang.org/u/Ender_L)\
**Post date:** [February 18, 2023, 12:52am UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/7 "2023-02-18T00:52:31Z")

</div>

Good to know. Thanks.  
And why does sum() sometimes not agree with itself either?

```julia
julia> function rowsum(A)
           out = Array{Float64,2}(undef, size(A, 1),1)
           for ii in axes(A, 1)
               out[ii,1] = sum(A[ii, :])
           end
           return out
       end
rowsum (generic function with 1 method)

julia> A=rand(10,1000);

julia> sum(A,dims=2)-rowsum(A)
10×1 Matrix{Float64}:
  2.2737367544323206e-13
  2.2737367544323206e-13
  4.547473508864641e-13
 -2.2737367544323206e-13
  7.389644451905042e-13
  1.7053025658242404e-13
 -2.8421709430404007e-13
 -2.2737367544323206e-13
 -5.115907697472721e-13
  5.684341886080802e-14

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 18, 2023, 3:50am UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/8 "2023-02-18T03:50:22Z")

</div>

because `sum(A, dims=2)` (when you have dims) is not implemented like that, it’s more like a naive sum

---

<div class="post-metadata">

**Author:** ![Ender\_L](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ender_l/32/47013_2.png) [@Ender\_L](https://discourse.julialang.org/u/Ender_L)\
**Post date:** [February 18, 2023, 7:05pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/9 "2023-02-18T19:05:42Z")

</div>

By naive sum do you mean like the loop I wrote at the start?

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [February 18, 2023, 7:15pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/10 "2023-02-18T19:15:03Z")

</div>

In the case where you have a large number of similar-magnitude integers (so their sum is `BigInt` territory) I think pairwise summation is asymptotically faster than sequential

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [February 18, 2023, 7:15pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/11 "2023-02-18T19:15:56Z")

</div>

> [@jling](#):
>
> it’s more like a naive sum

That is interesting to me. I would have thought that base sum always used Kahan summation, at least on items with known size. I wonder why that is not the case.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 18, 2023, 7:20pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/12 "2023-02-18T19:20:36Z")

</div>

Kahan summation is roughly 2x slower.

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [February 18, 2023, 7:27pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/13 "2023-02-18T19:27:10Z")

</div>

Ah sorry, I tend to always equate the term pairwise sum with Kahan sum. Anyways, what I meant is I’m wondering why summing along other dimensions would be any different than a standard sum.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 18, 2023, 7:42pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/14 "2023-02-18T19:42:25Z")

</div>

Not entirely sure. My guess would be that it’s just that it’s easier to write it as a maprecuce than writing the code to do pairwise summation over arbitrary dimensions.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 18, 2023, 8:00pm UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/15 "2023-02-18T20:00:54Z")

</div>

> [@Ender\_L](#):
>
> And why does sum() sometimes not agree with itself either?

In general, the reduction operations in Julia (`mapreduce`, `sum`, …) [are documented](https://docs.julialang.org/en/v1/base/collections/#Base.reduce-Tuple%7BAny,%20Any%7D) to treat the operator as associative, and to use an implementation-dependent associativity. If you want a particular associativity you have to use `foldl`/`foldr` or similar (or write your own loop).

---

<div class="post-metadata">

**Author:** ![Ender\_L](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ender_l/32/47013_2.png) [@Ender\_L](https://discourse.julialang.org/u/Ender_L)\
**Post date:** [February 21, 2023, 4:05am UTC](https://discourse.julialang.org/t/does-the-julia-intrinsic-sum-apply-fastmath-by-default/94815/16 "2023-02-21T04:05:00Z")

</div>

Good to know. I wasn’t aware this apply to not only `map` but also others.