# Comparing exp() performance on Julia versus numpy

**URL:** <https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325>\
**Category:** Performance\
**Created:** [August 5, 2022, 6:08am UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325 "2022-08-05T06:08:46Z")\
**Posts on this page:** 19\
**Page:** 2

<div class="post-metadata">

**Author:** ![suavesito](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/suavesito/32/34386_2.png) [@suavesito](https://discourse.julialang.org/u/suavesito)\
**Post date:** [August 5, 2022, 2:11pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/21 "2022-08-05T14:11:16Z")

</div>

Intel Core I7 6500U.

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [August 5, 2022, 2:14pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/22 "2022-08-05T14:14:07Z")

</div>

> [@jling](#):
>
> > [@torrance](#):
> >
> > Why isn’t `exp.(arr)` automatically applying simd optimizations?
> 
> we generally don’t do this.

That’s incorrect. For example, check out: `@code_llvm (^).(arr, 2)`. As @Oscar_Smith above also said, the broadcasting loop is being done in the context of the `@simd` macro.

Looking further into it, it seems `@simd` is not compatible with `exp` (and `sin`, `cos`), and the LoopVectorization is doing some magic to make these functions compatible.

I wonder if there’s any plans to upsteam these changes into base Julia to that `@simd` works with these standard math functions?

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [August 5, 2022, 2:16pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/23 "2022-08-05T14:16:00Z")

</div>

> [@torrance](#):
>
> I wonder if there’s any plans to upsteam these changes into base Julia to that `@simd` works with these standard math functions?

already done, see previous example and linked PR?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 5, 2022, 2:17pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/24 "2022-08-05T14:17:58Z")

</div>

This isn’t explicit optimization. What’s happening here is that for simple functions (ones small enough to be inlined), the compiler is able to automatically figure out vectorized versions. For a call like `exp.(v)` that is a much harder optimization to make, and getting optimal performance would definitely require a handwritten vectorized `exp` function (like the one in LoopVectorization).

---

<div class="post-metadata">

**Author:** ![suavesito](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/suavesito/32/34386_2.png) [@suavesito](https://discourse.julialang.org/u/suavesito)\
**Post date:** [August 5, 2022, 2:19pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/25 "2022-08-05T14:19:34Z")

</div>

The optimisation is done normally, but it depends on several things, LV do a lot of things more aggressively because it makes more assumptions and have some hand written implementation’s of functions.

Respect to `exp`, this two are related [#44865](https://github.com/JuliaLang/julia/pull/44865) and [#46338](https://github.com/JuliaLang/julia/pull/46238) which are in `master`.

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [August 5, 2022, 2:28pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/26 "2022-08-05T14:28:02Z")

</div>

> [@jling](#):
>
> already done, see previous example and linked PR?

Wow, yes actually - it’s about 4x times faster. Still not quite as fast as the 6x speedup of `@turbo` but a big difference. And I think (?) I can see simd instructions in the llvm output.

It doesn’t appear this has also been applied to `sin()` or `cos()` though.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 8, 2022, 12:08pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/27 "2022-08-08T12:08:21Z")

</div>

> [@torrance](#):
>
> It doesn’t appear this has also been applied to `sin()` or `cos()` though.

Does any language SIMD compile sin and cos by default (maybe Julia can (could [be made to]?) and should do that, but only if `@simd` applied, even if using intrinsics/functions)? It’s unclear if only available with Intel compiler or done by default, since not always better:

> <https://stackoverflow.com/questions/27933355/is-it-possible-to-get-multiple-sines-in-avx-sse>

> there is a vector version using SSE/AVX! **But the catch is that Intel C++ compiler must be used.**
> 
> This is called **Intel small vector math library** (intrinsics):
> 
> for 128bit SSE please use (double precision): **\_mm\_sin\_pd**
> 
> for 256bit AVX please use (double precision): **\_mm256\_sin\_pd**
> 
> The two intrinsics are actually very small functions consists of hand written SSE/AVX assemblies, and now you can process 4 sine calculations at once by using AVX :=) the latency is about ~10 clock cycles (if I remember correctly) on Haswell CPU.
> 
> By the way, the CPU needs to execute about 100 such intrinsics to warm up and reach its peak performance, if there is only a few sin functions needs to be evaluated, it’s better to use plain sin() instead.

> **[Laszlo - .NET Developer, Personal Blog](https://blog.ladeak.net/posts/math-sin-simd)**
>
> In this post I will look into how someone can implement Sin/Cosin functions with SIMD in NET 5. NET 5 does not have a wrapper on SIMD trigonometric functions, so it seems a good exercise to implement it.

> <https://stackoverflow.com/questions/36636159/where-is-clangs-mm256-pow-ps-intrinsic/36637424#36637424>

> […]
> 
> - **[Agner Fog’s VCL](https://www.agner.org/optimize/#vectorclass) has some math functions like `exp` and `log`**. (Formerly GPL licensed, now Apache).
> 
> - `https://github.com/microsoft/DirectXMath` (MIT license) - I think portable to non-Windows, and doesn’t require DirectX. […]

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [August 8, 2022, 1:38pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/28 "2022-08-08T13:38:09Z")

</div>

There is also [Sleef](https://github.com/shibatch/sleef). It supports ARM as well, so it is great.  
But from my experience, it is less performant than Intel SVML.  
I think SVML is now, to some degree, embodied in Clang and also in GCC.  
Yet for sure it is available for free with Intel OneAPI package.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [August 8, 2022, 1:52pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/29 "2022-08-08T13:52:10Z")

</div>

Related:

> **[GitHub - JuliaSIMD/SLEEFPirates.jl: Pirated SLEEF port.](https://github.com/JuliaSIMD/SLEEFPirates.jl)**
>
> Pirated SLEEF port. Contribute to JuliaSIMD/SLEEFPirates.jl development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![jebej](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jebej/32/1784_2.png) [@jebej](https://discourse.julialang.org/u/jebej)\
**Post date:** [August 8, 2022, 1:57pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/30 "2022-08-08T13:57:52Z")

</div>

It’s been hinted at already in the thread, but to be completely clear: here LoopVectorization uses a different, manually vectorized implementation of `exp`, this is why it’s faster, not because it’s optimizing the base julia `exp`.

It is likely that numpy has a similar vectorization built-it, hence why it is faster.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [August 8, 2022, 3:34pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/31 "2022-08-08T15:34:28Z")

</div>

@Oscar_Smith  
Julia `exp` regressed recently.  
Earlier, you could use call-site inlining to get it to vectorize, via `@inline exp(x[i])`, and it’d vectorize.  
This now no longer works because `pow_body` does not inline, and we need the entire function to inline.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 8, 2022, 3:36pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/32 "2022-08-08T15:36:38Z")

</div>

What do you mean? `exp` doesn’t call `pow_body`…

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [August 8, 2022, 3:36pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/33 "2022-08-08T15:36:40Z")

</div>

> [@jebej](#):
>
> It’s been hinted at already in the thread, but to be completely clear: here LoopVectorization uses a different, manually vectorized implementation of `exp`, this is why it’s faster, not because it’s optimizing the base julia `exp`.

It uses the same basic algorithm, has slightly higher error tolerance (around 1.5 ULP IIRC, instead of 0.5 for Base?)  
It does also have some special optimizations for AVX512 that the compiler won’t be able to figure out on its own.

But this is a problem with Julia’s compiler.  
Julia cannot vectorize code unless it is entirely inlined.  
Julia’s inliner also does not consider the possibility of vectorization.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [August 8, 2022, 3:37pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/34 "2022-08-08T15:37:07Z")

</div>

I have a non-inlined call to `pow_body` when looking at the typed code.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 8, 2022, 3:37pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/35 "2022-08-08T15:37:31Z")

</div>

Do you mean for floating point powers?

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [August 8, 2022, 3:39pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/36 "2022-08-08T15:39:40Z")

</div>

Yes

```julia
 • %82 = < constprop > pow_body(::Core.Const(2.0),::Core.Const(-53))::Float64

```

The constant 2.0^-53 prevents SIMD.  
Please do not rely on the compiler any more than necessary…

But I’ll rebuild the latest Julia master. Might be fixed with the effects analysis improvements.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 8, 2022, 3:48pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/37 "2022-08-08T15:48:50Z")

</div>

If so, that’s really dumb because it totally could just be `0x1p-53`

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [August 8, 2022, 4:10pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/38 "2022-08-08T16:10:15Z")

</div>

Yeah. It seems silly that a literal constant like `1e10` vs `10.0^10` or `0x1p-53` vs `2^-53` makes a big difference.  
But it’s been fixed on the latest master already.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [August 8, 2022, 4:12pm UTC](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325/39 "2022-08-08T16:12:15Z")

</div>

Actually, even without `@inline` things look really good on the latest Julia master!

```julia
julia> using LoopVectorization, BenchmarkTools

julia> function myfunc(arr)
           result = similar(arr)
           @simd for i in eachindex(arr)
               @inbounds result[i] = exp(arr[i])
           end
           return result
       end
myfunc (generic function with 1 method)

julia> function myfunc2(arr)
           result = similar(arr)
           @simd for i in eachindex(arr)
               @inbounds result[i] = @inline exp(arr[i])
           end
           return result
       end
myfunc2 (generic function with 1 method)

julia> function myfunc3(arr)
           result = similar(arr)
           @turbo for i in eachindex(arr)
               result[i] = exp(arr[i])
           end
           return result
       end
myfunc3 (generic function with 1 method)

julia> arr = rand(100_000);

julia> @btime myfunc($arr);
  76.586 μs (2 allocations: 781.30 KiB)

julia> @btime myfunc2($arr);
  65.682 μs (2 allocations: 781.30 KiB)

julia> @btime myfunc3($arr);
  67.174 μs (2 allocations: 781.30 KiB)

```

It’d be great if we could have regression tests to make sure it does not regress.

I’m not sure why `myfunc` is slower than `myfunc2`/would have to spend more time looking at it to see what is actually different.  
But it seems to inline on its own.

[Previous page](https://discourse.julialang.org/t/comparing-exp-performance-on-julia-versus-numpy/85325.md?page=1)
