# How to achieve multi-threaded vectorized FMA operations in the for-loop for SAXPY?

**URL:** <https://discourse.julialang.org/t/how-to-achieve-multi-threaded-vectorized-fma-operations-in-the-for-loop-for-saxpy/100805>\
**Category:** Performance\
**Tags:** question\
**Created:** [June 25, 2023, 8:38am UTC](https://discourse.julialang.org/t/how-to-achieve-multi-threaded-vectorized-fma-operations-in-the-for-loop-for-saxpy/100805 "2023-06-25T08:38:42Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![xinwu](https://avatars.discourse-cdn.com/v4/letter/x/a4c791/32.png) [@xinwu](https://discourse.julialang.org/u/xinwu)\
**Post date:** [June 25, 2023, 8:38am UTC](https://discourse.julialang.org/t/how-to-achieve-multi-threaded-vectorized-fma-operations-in-the-for-loop-for-saxpy/100805/1 "2023-06-25T08:38:42Z")

</div>

Hi everyone,

For serial vectorized FMA operations in the for-loop for SAXPY, one can use

```julia
@fastmath @inbounds @simd for i in 1:n

```

See also this discussion ([How to enable vectorized fma instruction for multiply-add vectors?](https://discourse.julialang.org/t/how-to-enable-vectorized-fma-instruction-for-multiply-add-vectors/100219)) and “The Julia Language V1.9.0” (see Page 434).

My question is: how to achieve multi-threaded vectorized FMA operations in the for-loop for SAXPY? The code snippet below gives error message:

```julia
@threads @fastmath @inbounds @simd for i in 1:n

```

The error message is:

```julia
ERROR: LoadError: ArgumentError: @threads requires a `for` loop expression

```

Thank you in advance!

PS.

- The solution using `@sync` and `@spawn` for such a problem, like SAXPY, is, IMHO, not optimal.
- I do not want to prefix every statements in the for-loop body with `@fastmath ...`. Possible for SAXPY, but definitely bad idea for a generic for-loop.

---

<div class="post-metadata">

**Author:** ![xinwu](https://avatars.discourse-cdn.com/v4/letter/x/a4c791/32.png) [@xinwu](https://discourse.julialang.org/u/xinwu)\
**Post date:** [June 25, 2023, 9:11am UTC](https://discourse.julialang.org/t/how-to-achieve-multi-threaded-vectorized-fma-operations-in-the-for-loop-for-saxpy/100805/2 "2023-06-25T09:11:12Z")

</div>

OK, find this Issue opened 4 years ago without any progress… [Cannot combine @simd and @threads on a loop · Issue #32684 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/32684)

Multithreading and vectorization are both critically important topics for performance!

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [June 25, 2023, 10:42am UTC](https://discourse.julialang.org/t/how-to-achieve-multi-threaded-vectorized-fma-operations-in-the-for-loop-for-saxpy/100805/3 "2023-06-25T10:42:03Z")

</div>

```julia
@threads for ...
   @fastmath begin
   ....
   end
end

```

Likely will do what you want. `@simd` doesn’t give you additional benefits over fast math here since it is primarily targeting reduction chains.