# Why doesn’t Julia fma automatically?

**URL:** <https://discourse.julialang.org/t/why-doesn-t-julia-fma-automatically/104982>\
**Category:** General Usage\
**Created:** [October 13, 2023, 3:31am UTC](https://discourse.julialang.org/t/why-doesn-t-julia-fma-automatically/104982 "2023-10-13T03:31:53Z")\
**Posts on this page:** 4\
**Page:** 3

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [October 17, 2023, 12:41am UTC](https://discourse.julialang.org/t/why-doesn-t-julia-fma-automatically/104982/41 "2023-10-17T00:41:26Z")

</div>

> [@ParadaCarleton](#):
>
> I think your post is the first one to bring up an actual case of these kinds of bugs causing problems in real life.

There were lots of concrete bogus examples in the other thread, I think this was one of the nicest ones:

> [@X \* y + z does not automatically use FMA instruction](https://discourse.julialang.org/t/x-y-z-does-not-automatically-use-fma-instruction/100640/19):
>
> Using FMA automatically can cause surprises. Consider the following: function f(a,b,c) @assert a\*b ≥ c return sqrt(a\*b-c) end a = 1.0 + 0.5^27 b = 1.0 - 0.5^27 c = 1.0 f(a,b,c) It works as expected. Now suppose the compiler automatically uses FMA inside f, like this: function f(a,b,c) @assert a\*b ≥ c return sqrt(fma(a,b,-c)) end Now f(a,b,c) throws and exception, because fma(a,b,-c) is negative, despite the assertion a\*b ≥ c.

And this was your “basic/case easy win `a*x + b`”

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [October 17, 2023, 2:33am UTC](https://discourse.julialang.org/t/why-doesn-t-julia-fma-automatically/104982/42 "2023-10-17T02:33:07Z")

</div>

> [@ParadaCarleton](#):
>
> Were the bugs all caused by people writing things like

```julia
function kernel(rho, T)
  P = rho[1] * T[1]

  if (abs(P - P) > 1e-16)
    @cuprintf("diff = %.16e\n", P - P)
  end
  nothing
end

rho = CuArray([1e-1])
T = CuArray([300.0])
@launch CUDA() kernel(rho, T, threads=1, blocks=1)

```

with the output

```julia
diff = 1.6653345369377348e-15

```

So `abs(P-P)` being different from `P-P`

```julia
	mul.f64 %fd4, %fd1, %fd3;
	neg.f64 %fd5, %fd4;
	fma.rn.f64 %fd2, %fd1, %fd3, %fd5;
	abs.f64 %fd6, %fd2;

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [October 18, 2023, 12:27pm UTC](https://discourse.julialang.org/t/why-doesn-t-julia-fma-automatically/104982/43 "2023-10-18T12:27:56Z")

</div>

> [@Oscar\_Smith](#):
>
> the problem is that compilers don’t work one line at a time. You basically only have 2 options: allow re-association and contraction or prevent them.

[clang allows finer-grained control](https://clang.llvm.org/docs/UsersManual.html#cmdoption-ffp-contract) than this — `-ffp-contract=on` will only fuse within a single statement in the source code, as allowed by the C99 standard, whereas `-ffp-contract=fast` will fuse across statements.

---

<div class="post-metadata">

**Author:** ![mikmoore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikmoore/32/31109_2.png) [@mikmoore](https://discourse.julialang.org/u/mikmoore)\
**Post date:** [October 18, 2023, 4:17pm UTC](https://discourse.julialang.org/t/why-doesn-t-julia-fma-automatically/104982/44 "2023-10-18T16:17:35Z")

</div>

Plugging this tangentially related [issue for selectable `@fastmath` flags](https://github.com/JuliaLang/julia/issues/49890) for those wishing to exercise restraint in their fastmath. We already have [`@simd` with less `@fastmath`](https://github.com/JuliaLang/julia/pull/49405) slated for v1.10 so that it isn’t [such a hazard](https://github.com/JuliaLang/julia/issues/49387).

[Previous page](https://discourse.julialang.org/t/why-doesn-t-julia-fma-automatically/104982.md?page=2)