Slow exp()?

Specific Domains Numerics
1 
julia> exp(1.3)
3.6692966676192444

julia> exp(1)
2.718281828459045

julia> 2.718281828459045 ^ 1.3
3.669296667619244

julia> @benchmark exp(1.3)
BenchmarkTools.Trial:
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     11.863 ns (0.00% GC)
  median time:      13.235 ns (0.00% GC)
  mean time:        13.247 ns (0.00% GC)
  maximum time:     72.282 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     999

julia> @benchmark 2.718281828459045 ^ 1.3
BenchmarkTools.Trial:
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     1.564 ns (0.00% GC)
  median time:      1.649 ns (0.00% GC)
  mean time:        1.721 ns (0.00% GC)
  maximum time:     12.501 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

Apart from a little inaccuracy, seems like “2.718281828459045 ^” is much faster than “exp()”. Similarly, “2.0 ^” is faster than “exp2()”, but exp() is particularly slow:

julia> @benchmark exp2(1.3)
BenchmarkTools.Trial:
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     5.700 ns (0.00% GC)
  median time:      5.730 ns (0.00% GC)
  mean time:        5.959 ns (0.00% GC)
  maximum time:     29.321 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

julia> @benchmark 2.0 ^ 1.3
BenchmarkTools.Trial:
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     1.564 ns (0.00% GC)
  median time:      1.572 ns (0.00% GC)
  mean time:        1.636 ns (0.00% GC)
  maximum time:     12.150 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

any suggestions?

2

Put it in functions:

julia> f() = exp(1.3);

julia> g() = 2.718281828459045 ^ 1.3;

julia> @btime f();
  1.815 ns (0 allocations: 0 bytes)

julia> @btime g();
  1.815 ns (0 allocations: 0 bytes)

Also, llvm is smart, so we are not actually doing a computation at all, just returning the answer:

julia> @code_llvm f()

; Function f
; Location: REPL[1]:1
define double @julia_f_35198() {
top:
  ret double 0x400D5AB83615F8B4
}
2 Likes
3 

julia> f(x) = exp(x)
f (generic function with 1 method)

julia> g(x) = 2.718281828459045^x
g (generic function with 1 method)

julia> using BenchmarkTools

julia> @btime f($1.3)
  11.023 ns (0 allocations: 0 bytes)
3.6692966676192444

julia> @btime g($1.3)
  91.612 ns (0 allocations: 0 bytes)
3.669296667619244
3 Likes
5

Actually, I can replicate this both on Julia 0.6 and 0.7. The code looks like this:

julia> @code_llvm f(1.3)

define double @julia_f_63011(double) #0 !dbg !5 {
top:
  %1 = call double @julia_exp_62847(double %0)
  ret double %1
}

julia> @code_llvm g(1.3)

define double @julia_g_62927(double) #0 !dbg !5 {
top:
  %1 = call double @llvm.pow.f64(double 0x4005BF0A8B145769, double %0)
  %2 = fadd double %0, 0x4005BF0A8B145769
  %notlhs = fcmp ord double %1, 0.000000e+00
  %notrhs = fcmp uno double %2, 0.000000e+00
  %3 = or i1 %notrhs, %notlhs
  br i1 %3, label %L9, label %if

if:                                               ; preds = %top
  call void @jl_throw(i8** inttoptr (i64 140113430158408 to i8**))
  unreachable

L9:                                               ; preds = %top
  ret double %1
}

In other words, exp() is fast. :slight_smile:

6

Sort of “related” to this (and it’s Friday, so cut me some slack. :slight_smile:).

Computing log10(x) using log(x)*log10(exp(1)), with log10(exp(1)) replaced by the resulting “numerical value”, is faster than log10(x) according to the benchmark tools (if I’m using them properly?). Obviously not producing exactly the same value, but sometimes speed trumps accuracy… Julia 1.0.0/Windows 7.

using BenchmarkTools

f1(x) = log(x)*0.4342944819032518

@btime log10($1.3)
  22.895 ns (0 allocations: 0 bytes)
0.11394335230683679

@btime f1($1.3)
  8.684 ns (0 allocations: 0 bytes)
0.11394335230683678
7

Hm, how can you see that exp is fast from this? In the LLVM code there it just calls another function.

1 Like
8

amazing. thanks. I’ve learned a lot.

9

Yes, this is something that is useful to keep in mind. Generally, if a function is called something like log10 (or exp or …) it will be implemented as fast as possible, but within the constraints of being within 1ulp of the “true solution”. That is, not quite correctly rounded, but quite close still. I’m not sure what the worst case error is for your f1, but I’m certain that it is not 1ulp.

That said, we do have a @fastmath macro, and there’s certainly more we could do with that, to actually make it choose faster less accurate versions for all the elementary functions.

2 Likes
10

sorry, I’m confused.

what’s @fastmath? (I cannot find docs about it). Also, I see Base.FastMath, what is it???

I mean, if there exists faster maths, why keep the original (slower) maths ???

11

faster math means in some cases less accurate math.

12

That’s interesting. It was quite easy for me to find the documentation. How’d you look?

in the docs search functionality
https://docs.julialang.org/en/v1.0.0/search/?q=fastmath

and in the repl

weird
weird972Ă—338 9.9 KB

13

I’m using JuliaPro 0.6.3 … and somehow there is no help on @fastmath ???

help?> @fastmath
No documentation found.
Base.FastMath.@fastmath is a macro.
# 1 method for macro "@fastmath":
@fastmath(expr::ANY) in Base.FastMath at fastmath.jl:127

so, starting julia --math-mode=fast would put all maths in fastmath mode?

14

I think the docs were added in 0.7.