Output distribution of rand(Float32) and rand(Float64), thread 2

[mbauman]

@mbauman

Where the line for “good enough” is drawn will obviously depend on what you’re doing with the random numbers. So what are some example lines in the sand? Are there algorithms that would have significant error with 2^{-53}2−532^{-53} but not with some 2^x2x2^x?

So to answer my own question here — doing any sort of thresholding (like that Bernoulli example way up above) will “snap” its effective threshold to the next-highest representable value in the output. So, testing rand(Float32) < 1e-8 is effectively the same as thresholding against ceil(1e-8*2^24)/2^24 ≈ 5.96e-8 — there’s that factor of 6. So this means any threshold x may have an error of up to \frac{x + 2^{-24}}{x}: That’s ~6% for values on the order of 10^{-6}, 0.6% for 10^{-5}, 0.06% for 10^{-4}. Now we’re getting into very common territory.

So that’s pretty conclusive to me that we definitely need to be doing better for rand(Float32). The exact same analysis holds for all those other “density” values — so for with a Float64-status-quo density of steps of 2^{-53} we’re talking about errors of 0.0001% for 10^{-10}.