# BetterExp 0.1.0

**URL:** <https://discourse.julialang.org/t/betterexp-0-1-0/44747>\
**Category:** Package Announcements\
**Tags:** math\
**Created:** [August 11, 2020, 3:49pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747 "2020-08-11T15:49:40Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 11, 2020, 3:49pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/1 "2020-08-11T15:49:40Z")

</div>

Do you use `exp`? Do you want it to be faster and (slightly) more accurate? If so, use [GitHub - oscardssmith/BetterExp.jl: A faster and more accurate Exp (and Exp2, Exp10) functions for Julia.](https://github.com/oscardssmith/BetterExp.jl). It adds versions of `exp`, `exp2` and `exp10` that are 2x faster than base for `Float64`, and 3x faster for `Float32` than base.

Edit: Hopefully this will make it’s way into Base soon enough, but in the meantime, this will hopefully be useful.

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [August 11, 2020, 3:53pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/2 "2020-08-11T15:53:20Z")

</div>

Do you have a reference for which algo this implements versus what Base uses? Or is it the same thing just tweaking the Julia implementation to be faster?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 11, 2020, 3:59pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/3 "2020-08-11T15:59:54Z")

</div>

Most of the speed difference is that these algorithms only have 1 branch in the normal case. (The branch is for answers with subnormal, 0, Inf or Nan outputs). This both speeds things up, but also allows for automatic vectorization.  
For Float32, the algorithm is pretty similar to base, except it uses a 7 term polynomial instead of a much shorter rational function.  
The `Float64` version is fairly different. It uses a table with 256 elements, to reduce the range the polynomial kernel needs to approximate. This allows a much smaller polynomial to maintain full accuracy.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 11, 2020, 4:03pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/4 "2020-08-11T16:03:16Z")

</div>

The `Float64` version also has a bunch of minor tricks to get full precision with minimal use of double-double arithmetic. The two main tricks are storing extra bits in the table, and having the kernel approximate `expm1` instead of `exp`.

---

<div class="post-metadata">

**Author:** ![jmert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmert/32/3161_2.png) [@jmert](https://discourse.julialang.org/u/jmert)\
**Post date:** [August 11, 2020, 4:13pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/5 "2020-08-11T16:13:16Z")

</div>

Can you share how the coefficients for the `expm1b_kernel` have been chosen? For the `::Val{ℯ}` case, I can recognize off-hand that those aren’t quite the Taylor series coefficients, so I’m guessing they’ve been numerically optimized in some way.

(I’ve been silently following the Julia PR, and I got sucked into trying to understand where all the constants were coming from. So far I have not been able to get a minimax optimization to reproduce those specific coefficients, so I’m curious if there’s some extra constraints I haven’t been able to reverse engineer yet.)

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 11, 2020, 4:18pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/6 "2020-08-11T16:18:27Z")

</div>

The coefficients are minimax polynomials obtained by `Remez.jl`. The thing that’s a little tricky is that since I want the first coefficient to be `0`, I actually approximate `x -> x == 0 ? one(x) : expm1(x)/x`. Then to make it so the error minimization is absolute error with respect to `expm1`, you need to weight the error function by a factor of `x`. Thus the full way to generate the weights (for `::Val{ℯ}`) ends up being

```julia
ratfn_minimax(x -> x == 0 ? one(x) : expm1(x)/x, (-log(2)/512,log(2)/512), 3, 0,(x,y) -> x)

```

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 11, 2020, 4:22pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/7 "2020-08-11T16:22:16Z")

</div>

One thing worth looking into (especially for `Float32`) is if rounding these coefficients means that the error they generate increases significantly. If so, it might be worth doing an exhaustive search of nearby coefficients to see if any combination gives better results.

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [August 11, 2020, 4:25pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/8 "2020-08-11T16:25:48Z")

</div>

Couldn’t find the PR that adds this to Base, can you please paste it here for reference?

---

<div class="post-metadata">

**Author:** ![jmert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmert/32/3161_2.png) [@jmert](https://discourse.julialang.org/u/jmert)\
**Post date:** [August 11, 2020, 4:26pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/9 "2020-08-11T16:26:06Z")

</div>

[https://github.com/JuliaLang/julia/pull/36761](https://github.com/JuliaLang/julia/pull/36761)

---

<div class="post-metadata">

**Author:** ![jmert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmert/32/3161_2.png) [@jmert](https://discourse.julialang.org/u/jmert)\
**Post date:** [August 11, 2020, 4:35pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/10 "2020-08-11T16:35:21Z")

</div>

OK, thank you for the clarification. I see now that given your weight function, that’s a minimax of the absolute error — I’d been playing around with minimaxing the relative error (so my weight function was `x -> iszero(x) ? one(x) : x / expm1(x)`). I also coded up my own minimax algorithm (as a way to teach myself the principles), so I’m guessing part of the remaining residual difference is just some implementation details on my part.

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [August 11, 2020, 5:52pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/11 "2020-08-11T17:52:37Z")

</div>

Do you have a guarantee on the accuracy of the new versions? Do they give exactly the same results as the current implementation?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 11, 2020, 5:57pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/12 "2020-08-11T17:57:28Z")

</div>

They don’t give exactly the same results, but the `Float64` versions have a maximum error of `.76` ULPs compared to `.88` ULP for base. For Float32, my versions have a max error of `1` ULP compared to `.9` for base.

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [August 11, 2020, 6:14pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/13 "2020-08-11T18:14:44Z")

</div>

Thanks. Could you please give a couple of examples of the worst cases?  
How did you find them and what did you compare with?

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [August 11, 2020, 6:15pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/14 "2020-08-11T18:15:09Z")

</div>

Is it exactly 1 ulp for Float32? This is important to know for directed rounding purposes.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 11, 2020, 6:28pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/15 "2020-08-11T18:28:58Z")

</div>

It was 1.008 for `exp2`, but I’m pushing a new commit that brings it down to .87 max. I somehow was missing a degree for `exp2`.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 11, 2020, 10:12pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/16 "2020-08-11T22:12:14Z")

</div>

Does interval arithmetic provide an easy way to get the maximum error of a polynomial (in ULPs)? If so, how would I do it? I’m thinking of trying an exhaustive search for better polynomials than the rounded minimax polynomials, but exhaustive checking is too slow.

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [August 11, 2020, 10:15pm UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/17 "2020-08-11T22:15:14Z")

</div>

Good question. I don’t think it’s too easy. People use tools like Sollya and Gappa for this I believe.

---

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [August 12, 2020, 12:41am UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/18 "2020-08-12T00:41:32Z")

</div>

Can we directly get _expm1_?  
I mean exp(x)-1

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [August 12, 2020, 12:48am UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/19 "2020-08-12T00:48:50Z")

</div>

`sollya` is the tool that some people use to get there.  
The “onramp” is steep. Here are some resources:

[Sollya Users Manual](http://sollya.gforge.inria.fr/sollya-weekly/sollya.pdf)  
[Sollya Commands](http://sollya.gforge.inria.fr/sollya-weekly/help.php)

- look for examples of sollya code

Rigorous Polynomial Approximations and Applications  
[Rigorous Polynomial Approximations and Applications (slides)](http://homepages.laas.fr/mmjoldes/Exposes/exposeThese.pdf)

- sollya has much of what follows built-in  
see the sollya commands `fpminimax` `implementpoly`

[Optimizing Polynomials for floating point implementation](https://arxiv.org/pdf/0803.0439.pdf)  
[Exchange algorithm for … error-optimized polynomials](https://hal.archives-ouvertes.fr/hal-02006606/document)

- also of interest

[Accurate Horner Form Approximations to Special Functions](https://arxiv.org/pdf/1508.03211.pdf)

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 12, 2020, 2:21am UTC](https://discourse.julialang.org/t/betterexp-0-1-0/44747/20 "2020-08-12T02:21:08Z")

</div>

Do you mean get a faster method for `expm1(x)`? I don’t have one done yet, but plan to. The reason `expm1b` kernel isn’t good enough is that it’s designed to only be accurate enough when you add 1, which wipes out a lot of the least significant bits (at least 10). Since `expm1` has outputs close to 0, it is much harder to approximate within 1 ULP.

[Next page](https://discourse.julialang.org/t/betterexp-0-1-0/44747.md?page=2)