I do want to point out that if your wasn’t written with any particular order in mind, @fastmath will often be more accurate, e.g. it will use fma instead of * followed by + on computers with fma-support, which incurs fewer rounding steps.
Or, using the problematic example here, 1.0 * 369.3299304675746 + 6.755399441055744e15 - 6.755399441055744e15 == 369.3299304675746 if rounding weren’t a problem. So in some sense, the fastmath version – by returning 369.3299304675746 instead of 369.0 – is more accurate here.
However, 369.0 was the intended result.
The exp code was deliberately written with floating point behavior and rounding in mind. The IEEE version does precisely what the author intended.
So I would suggest that if you didn’t have specific intentions about order or rounding behavior – e.g. the order you specified was picked out of all possibilities out of convenience – then switching from one arbitrary order to another specified by @fastmath will increase accuracy more often than not.
But when these were chosen deliberately, i.e. not arbitrarily, results can be catastrophic for accuracy. Starting Julia with --math-mode=fast:
julia> x = randn(10)'
1×10 adjoint(::Vector{Float64}) with eltype Float64:
0.411952 -1.20621 -0.780723 0.69826 -1.60801 -1.88011 0.363784 0.663141 -0.874339 0.613377
julia> sinpi.(x)
1×10 Matrix{Float64}:
0.0 -0.0 -0.0 0.0 -0.0 -0.0 0.0 0.0 -0.0 0.0
julia> cospi.(x)
1×10 Matrix{Float64}:
1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
I think it’d be nice to use fastmath in some more places, e.g. in (add/mul)_fast(::ForwardDiff.Dual, ...) to optionally avoid some of the multiply-by and add-zeros from ForwardDiff that require some of fast flags to actually eliminate.
But it shouldn’t be applied to code written by others who may have implemented algorithms requiring specific sequences of operations.
Global fastmath is pretty bad IMO.
Which is part of the reason I’m somewhat encouraging applying it (or the safer alternative: @muladd) more locally: get rid of the incentive to use it globally by having the places that can use it safely already benefit from fma instructions.
EDIT: I think @muladd and @simd should be preferred over @fastmath, as the two of them cover the most important optimizations enabled by @fastmath with less of the safety concerns (e.g., handling of NaNs).