Help to find package for DSA (Discrete Stochastic Arithmetic) implementation

Specific Domains Numerics
,
1

Hi, I’m working in a project where we use DSA method (based-on CESTAC) to validate results of numerical software. Currently, I’m using Cadna (written in C) to solve this problem. The advantage of Cadna is it has example for CUDA implementation instead of just CPU like other tools.
I’m looking for a package in Julia that match my requirements:

  • Good CUDA support for half and bfloat16
  • Implement DSA or relevant mathematical approach. (I’m still new to DSA so I guess, there is other techniques to solve numerical validation problem).

The package StochasticArithmetic.jl seems not maintain anymore and only support SFloat64. Is there any expert out there have experience in this field? I really need an suggestion, thank you.

2

Hi, I’d be happy to revive StochasticArithmetic.jl if you have some interest in it. However, I don’t know much about GPUs, and haven’t the slightest clue how to implement DSA (or any form of Stochastic Arithmetic) in a GPU-compatible way.

I’d advise you to also look at @milankl’s work, especially StochasticRounding.jl. It’s not exactly DSA/CESTAC, but I tend to favour this kind of stochastic arithmetic these days, and this ecosystem is much more alive than my own StochasticArithmetic.jl. I don’t know whether StochasticRounding.jl supports GPUs, though.

2 Likes
3

I don’t know whether StochasticRounding.jl supports GPUs, though.

I haven’t tested whether or which issues arise on a GPU, if someone can test it, and open issues in StochasticRounding.jl then I’d be happy to address them!

4

But yes, to clarify, StochasticRounding.jl only uses the a distance-proportional rounding mode where x between a and b > a is stochastically rounded to b at chance p = (x-a) / (b-a) and to b at chance 1-p. So like if x is 1/4 of the way between a and b then there’s a 1/4 chance that it’s rounded to b. This rounding mode is exact in expectation, meaning that the average rounding error decreases for an increasing number of arithmetic operations.

5

What is the theory or paper that you implement in your package?

6

StochasticArithmetic.jl implements Vignes’ CESTAC arithmetic[1].

https://ffevotte.github.io/StochasticArithmetic.jl/dev/

In comparison to DSA:

  • CESTAC is asynchronous (whereas DSA is synchronous)
  • you can do as many evaluations as you like (whereas DSA fixes 3 evaluations)

In contrast, I think StochasticRounding.jl implements an arithmetic more similar to Stott Parker’s Monte Carlo Arithmetic[2] (MCA): instead of rounding upwards or downwards with probability 1/2 (like in CESTAC), the probability is adjusted so that the expectation of the stochastic result matches the real, mathematical result.

See e.g. this paper for a comprehensive comparison of CESTAC/DSA vs MCA (Disclaimer: I’m one of the authors)

Devan Sohier, Pablo De Oliveira Castro, François Févotte, Bruno Lathuilière, Eric Petit, and Olivier Jamond. Confidence intervals for stochastic arithmetic. ACM Transactions on Mathematical Software, 47(2), 2021.

  1. Jean Vignes and Michel La Porte. Error Analysis in Computing, 1974 ↩︎

  2. Douglas Stott Parker. Monte carlo arithmetic: exploiting random-
    ness in floating-point arithmetic. Technical Report CSD-970002,
    UCLA Computer Science Dept., 1997. ↩︎

1 Like