# Use different methods depending on presence of FMA

**URL:** <https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533>\
**Category:** Internals & Design\
**Created:** [June 29, 2017, 5:01am UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533 "2017-06-29T05:01:51Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [June 29, 2017, 5:01am UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/1 "2017-06-29T05:01:51Z")

</div>

Say you’re doing argument reduction for trig functions for double floats (wonder where this comes from…), then you should use different algorithms depending on the magnitude of your argument. Therefore, you end up with some branching and special cases depending on the required precision (say, around multiples of pi/2 or for numbers larger than 2.0^20).

Now, let’s add FMA to the mix. Due to the fact that FMAs are calculated with just one final rounding, we can in some sense calculate the above to higher precision for a given algorithm. However, given this increased precision, it no longer makes sense to switch to the more expensive versions of the algorithms near pi/2 multiples, and we can wait longer to switch to the expensive Payne Hanek scheme. The reduction scheme changes quite a lot.

In [https://github.com/JuliaLang/julia/pull/22603](https://github.com/JuliaLang/julia/pull/22603) I do not use FMAs, but many systems have these instruction sets available these days, and it would be stupid not to take advantage of this. However, I am unsure how to best go about this. Do you check for the presence of FMA instructions when `make`ing and include either of the two depending on a boolean flag, or?

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [June 29, 2017, 6:09am UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/2 "2017-06-29T06:09:48Z")

</div>

[https://github.com/JuliaLang/julia/issues/9855](https://github.com/JuliaLang/julia/issues/9855)

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [June 29, 2017, 6:12am UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/3 "2017-06-29T06:12:01Z")

</div>

Perhaps I am missing something, but I thought that `Base.fma` always does, well, FMA, just that it is done in software when not supported by the CPU. AFAIK the latter is available on all “recent” (\< 5 yo) Intel and AMD CPUs, and quite a few ARM ones as well (Raspberry Pi is perhaps an exception).

If your concern is speed on platforms that don’t support native FMA, you could check `Sys.ARCH` and use the FMA version of your algorithm only when it is `:x86_64`. You would miss a few cases though, but this would cover a lot.

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [June 29, 2017, 7:09am UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/4 "2017-06-29T07:09:43Z")

</div>

The problem is not speed, it’s the fact that if you have FMA you can use a better approach that doesn’t have to jump through as many hoops. For example, you can avoid using the 3 split version of Cody Waite for the medium case, and you can use Cody Waite for larger values than 2.0^20. So, as I tried to explain above, I want to distinct methods to be used depending on whether Julia was built on a system with or without fma.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [June 29, 2017, 7:18am UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/5 "2017-06-29T07:18:59Z")

</div>

Then I am confused — I thought that FMA was always available, via `Base.fma`. At least that’s what the documentation says.

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [June 29, 2017, 7:25am UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/6 "2017-06-29T07:25:18Z")

</div>

Alright, then let me be 110% explicit. If you have _hardware_ FMA support, that’s why I used the word “instruction” above.

If you don’t have hardware fma support then `fma(x,y,z)` can be rather slow, so you want to use `muladd(x,y,z)`, but again, this is not what I am after. If I write `muladd` on a system without hardware FMA and use the approach I asked for above, then I will get incorrect results. If I use `fma` on a system without hardware FMA I can use the approach above, but there’s a good chance it will be _slower_, and then I should have used the original reduction scheme. This is why I want to use two different _implementations_ based on hardware FMA being available or not.

I know most modern systems should have it, but I don’t think it makes sense to have old systems have a) an incorrect reduction scheme or b) an unnecessarily slow reduction scheme.

Edit: basically, we are repeating the discussion in the link Per posted…

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [June 29, 2017, 7:49am UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/7 "2017-06-29T07:49:09Z")

</div>

I’m facing the same problem. At the moment, I’m simply assuming that `Base.fma` is fast, and don’t bother about the fact that a better algorithm should be used when hardware fma is not available. This approach works well on 100% of the systems that I’ve tested my code on. 😉

It’d be really neat to be able to do `@static if has_fast_fma(T) alg1() else alg2() end`, but so far I’ve been too lazy to write that function myself.

---

<div class="post-metadata">

**Author:** ![tkelman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkelman/32/692_2.png) [@tkelman](https://discourse.julialang.org/u/tkelman)\
**Post date:** [June 29, 2017, 12:24pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/8 "2017-06-29T12:24:04Z")

</div>

You can inquire about what cpu type llvm and openblas think your processor / system image are. There’s also a jl\_cpuid for the raw data. Yichao has a PR open that adds a lot more of this functionality.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [June 29, 2017, 8:13pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/9 "2017-06-29T20:13:25Z")

</div>

Note that it is never a good idea to query feature support from could at runtime, it will be extremely slow.

And yes, I do want to add user API for CPU dispatch too, not sure what it will look like and it’ll probably not be in the current pr.

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [June 29, 2017, 8:39pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/10 "2017-06-29T20:39:15Z")

</div>

> [@yuyichao](#):
>
> Note that it is never a good idea to query feature support from could at runtime, it will be extremely slow.

I’m not sure if I didn’t really explain myself that well, but this is _exactly_ the thing I want to avoid. I want this to be settled at build time.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [June 30, 2017, 6:15am UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/11 "2017-06-30T06:15:24Z")

</div>

Well, that’s replying to Tony’s comment.

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [June 30, 2017, 12:07pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/12 "2017-06-30T12:07:28Z")

</div>

How about something like this?

```julia
"Predicate for testing if fast fused-multiply-add is available for the given type on this system."
has_fast_fma(::Type) = false
for T in [Float32, Float64]
  if contains(sprint(code_native, fma, (T, T, T)),"vfmadd")
    @eval has_fast_fma(::Type{$T}) = true
  end
end

```

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [June 30, 2017, 12:46pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/13 "2017-06-30T12:46:16Z")

</div>

What does it supposed to do?

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [June 30, 2017, 12:58pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/14 "2017-06-30T12:58:45Z")

</div>

It creates the function `has_fast_fma` that I referred to earlier. (Obviously this is a quick example, and not intended to be comprehensive.)

The idea is to be able to write code like:

```julia
@static if has_fast_fma(Float64)
  algorithm 1, which uses fma
else
  algorithm 2, which does does the same thing, without using fma
  and which is faster than algorithm 1 would be with software fma.
end

```

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [June 30, 2017, 1:26pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/15 "2017-06-30T13:26:16Z")

</div>

I mean what’s the semantics you want?  
If you just want a flag that works on only one host, just checking the result of `muladd` should work. Checking the assembly is much less relyable.  
If you want a way to make the compiled code portable, you can use a non-const Bool flag if you can afford a branch. If you don’t want the branch, it’s not possible ATM.

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [June 30, 2017, 2:02pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/16 "2017-06-30T14:02:59Z")

</div>

OK, so something like this?

```julia
has_fast_fma(::Type{T}) where {T} = muladd(one(T)+eps(T),-one(T)+eps(T),one(T)) == eps(T)*eps(T)

```

I don’t need to make the compiled code portable. (I didn’t even know that Julia supported creating compiled libraries.) I intend to use this in a module which is distributed as source code and (pre)compiled on the host where the code runs.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [June 30, 2017, 2:10pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/17 "2017-06-30T14:10:25Z")

</div>

Better make it a constant. LLVM constant folds `muladd`.

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [June 30, 2017, 2:15pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/18 "2017-06-30T14:15:02Z")

</div>

Would this work?

```julia
@generated has_fast_fma(::Type{T}) where {T} = muladd(one(T)+eps(T),-one(T)+eps(T),one(T)) == eps(T)*eps(T)

```

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [June 30, 2017, 2:26pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/19 "2017-06-30T14:26:08Z")

</div>

That’s worse.

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [June 30, 2017, 3:20pm UTC](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533/20 "2017-06-30T15:20:42Z")

</div>

Why is that worse? I thought the `@generated` macro would call the `muladd` function (rather than inlining and constant-folding it) and the macro would then return a constant `Bool` as the method body. Maybe I’ve misunderstood how `@generated` works.

I tried running the code on both a computer with fma and one that doesn’t have it. The results form the `@generated` function seem correct on both systems. (Without `@generated`, the function returns `true` even on the non-fma system.)

[Next page](https://discourse.julialang.org/t/use-different-methods-depending-on-presence-of-fma/4533.md?page=2)