Mean of integers overflows - bug or expected behaviour?

This seems like a circular argument; that’s an algorithm, and you could assert it as a definition, but I would say that it is not the conventional definition. You could just as well “define” norm(x) = sqrt(sum(abs2, x)) and say that Inf is therefore the “correct” result for norm([1e300]). Put another way, if we change the implementation of sum in a way that changes the roundoff error for floating-point inputs, are we changing the “definition” of mean(::Vector{Float64})?

For a floating-point function, it is the ordinary expectation to define “correctness” and “accuracy” with respect to the function as computed with infinite precision (i.e. over ℝ). I think that expectation applies here, since mean(::Vector{Int}) returns a Float64.

(Matters are different for arithmetic functions whose inputs and outputs are a finite-precision integer type.)

6 Likes