# Optimizing sums of products (dot products)

**URL:** <https://discourse.julialang.org/t/optimizing-sums-of-products-dot-products/119515>\
**Category:** Performance\
**Tags:** simd, linearalgebra, sum\
**Created:** [September 17, 2024, 4:40pm UTC](https://discourse.julialang.org/t/optimizing-sums-of-products-dot-products/119515 "2024-09-17T16:40:15Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [September 18, 2024, 12:53am UTC](https://discourse.julialang.org/t/optimizing-sums-of-products-dot-products/119515/5 "2024-09-18T00:53:46Z")

</div>

> [@xiaodai](#):
>
> Or is there an implicit assumption that floats are approximations anyway, so there is likely to be some small residuals of unmatched.

In some algorithms you can re-order things without any big problem (this is why many functions like `reduce` and `sum` in Julia leave their associativity/ordering/algorithm unspecified). But in others it would lead to catastrophe, or at least undesirable behavior.

The compiler doesn’t know which is which, so it defaults to doing what computer languages are usually supposed to do:

> “The good news about computers is that they do what you tell them to do. The bad news is that they do what you tell them to do.” — Ted Nelson

The compiler is not allowed to change the _results_ of your code, even if it _thinks_ the change is small (it isn’t always!). You don’t want a compiler that avoids disastrously rewriting your code only 99% of the time.

That’s why usually languages require some flag like `@simd` or `@fastmath` to give them explicit permission to re-order/re-associate floating-point operations in ways that might change the result. This is not unique to Julia, e.g. [gcc has `-ffast-math`](https://gcc.gnu.org/wiki/FloatingPointMath) to enable “unsafe” floating-point optimizations.

> [@xiaodai](#):
>
> Case in point, I was taught in numerical compute to compare floats like this `abs(a-b) < machine_epsilon` or something like.

Note that this is wrong — the [machine epsilon](https://en.wikipedia.org/wiki/Machine_epsilon) is a _relative_ tolerance, not an absolute tolerance. This is a common confusion. See also (e.g.) [History of `isapprox` in languages - #14 by stevengj](https://discourse.julialang.org/t/history-of-isapprox-in-languages/119347/14)

(Nor will different algorithms generally agree to a relative tolerance of the machine epsilon either … you often lose more digits than that. And sometimes you are _more_ accurate than that. Precision ≠ accuracy.)

---

_[View the full topic](https://discourse.julialang.org/t/optimizing-sums-of-products-dot-products/119515)._