# Mean of integers overflows - bug or expected behaviour?

**URL:** <https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767>\
**Category:** Statistics\
**Tags:** integer-overflow\
**Created:** [February 17, 2020, 4:54pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767 "2020-02-17T16:54:34Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [February 18, 2020, 4:00pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/21 "2020-02-18T16:00:22Z")

</div>

Raw A2D converter output is indeed an integer, but it represents a position on a fixed scale. The Julian approach to measurement data is FixedPointNumbers.jl, which already does the safe thing here:

```julia
julia> mean(rand(N0f32, 1000))
0.5003701349572203

```

---

<div class="post-metadata">

**Author:** ![Mikhail\_Kagalenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikhail_kagalenko/32/13257_2.png) [@Mikhail\_Kagalenko](https://discourse.julialang.org/u/Mikhail_Kagalenko)\
**Post date:** [February 18, 2020, 4:00pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/22 "2020-02-18T16:00:32Z")

</div>

> [@braamvandyk](#):
>
> Horses for courses. I deal with thousands of different measurements every day in my work. Temperatures, pressures, flows, compositions, reaction rates etc. In more than 20 years I have yet to find one of them that was an integer.

Show me a single ADC converter that outputs floating point values.

---

<div class="post-metadata">

**Author:** ![braamvandyk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/braamvandyk/32/5086_2.png) [@braamvandyk](https://discourse.julialang.org/u/braamvandyk)\
**Post date:** [February 18, 2020, 4:01pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/23 "2020-02-18T16:01:44Z")

</div>

Show me a single thermocouple, flow orifice, pressure gauge, or GC that outputs integers. As I said, horses for courses.

---

<div class="post-metadata">

**Author:** ![Mikhail\_Kagalenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikhail_kagalenko/32/13257_2.png) [@Mikhail\_Kagalenko](https://discourse.julialang.org/u/Mikhail_Kagalenko)\
**Post date:** [February 18, 2020, 4:09pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/24 "2020-02-18T16:09:21Z")

</div>

> [@braamvandyk](#):
>
> Show me a single thermocouple, flow orifice, pressure gauge, or GC that outputs integers.

If they are interfaced to a computer, all of them output integers, whether you realize it or not. For many examples of thermocouples you consult the [site](https://www.analog.com/en/products/analog-to-digital-converters/integrated-special-purpose-ad-converters/digital-temperature-sensors.html) of one of the largest vendors.  
.

---

<div class="post-metadata">

**Author:** ![braamvandyk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/braamvandyk/32/5086_2.png) [@braamvandyk](https://discourse.julialang.org/u/braamvandyk)\
**Post date:** [February 18, 2020, 4:10pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/25 "2020-02-18T16:10:39Z")

</div>

And if you deal directly with the A2D output, then that is your reality. I deal with the temperature measurement. That is my reality. My values are floats.

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [February 18, 2020, 4:11pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/26 "2020-02-18T16:11:02Z")

</div>

Unless you’re using an analog gauge, (and thanks to the profusion of silicon, few of us are), all these voltage/charge/current signals will be converted to fixed-point integers by an ADC somewhere between the sensor and your data acquisition system. I just installed a bunch of Honeywell pressure gauges with an onboard chip that reads the strain bridge measurement as a voltage, converts it to digital with an ADC, performs signal-conditioning, and converts it back to an analog voltage level for output. My data acquisition system then takes that analog voltage, converts it back to digital, and performs additional signal conditioning. The amount of back-and-forth is ugly, but none of us at the interface between computers and measurement equipment can ignore the reality of signal discretization.

---

<div class="post-metadata">

**Author:** ![braamvandyk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/braamvandyk/32/5086_2.png) [@braamvandyk](https://discourse.julialang.org/u/braamvandyk)\
**Post date:** [February 18, 2020, 4:12pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/27 "2020-02-18T16:12:00Z")

</div>

Of course, but I suspect very few of us are calculating the mean of the raw A2D output, rather than the resulting float value…

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [February 18, 2020, 4:14pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/28 "2020-02-18T16:14:25Z")

</div>

It really depends on the measurement system in question. Especially for high-bandwidth signals like audio/imagery, there’s a significant advantage to be gained by representing (and storing) signals using only the number of bits required.

---

<div class="post-metadata">

**Author:** ![improbable22](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/improbable22/32/5464_2.png) [@improbable22](https://discourse.julialang.org/u/improbable22)\
**Post date:** [February 18, 2020, 4:28pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/29 "2020-02-18T16:28:00Z")

</div>

While I agree that at some level an A2D gives you an integer, do they even make them with 64-bit ranges? (I think that may be more accurate than any number known to mankind?)

From say 24 bits there would be a lot of headroom. Does `mean(::Vector{Int32})` accumulate in Int64? Yes, it seems to:

```julia
julia> i32 = abs.(rand(Int32, 10000));

julia> mean(i32) == mean(float, i32) == mean(Int64, i32)
true

julia> i64 = abs.(rand(Int, 10000));

julia> mean(i64) - mean(float, i64)
-4.63382211131584e18

```

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [February 18, 2020, 4:50pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/30 "2020-02-18T16:50:47Z")

</div>

The behavior of `sum` (and `mean`, by extension) depends on whether your system’s word size:

```nohighlight
help?> sum
  sum(f, itr)

  Sum the results of calling function f on each element of itr.

  The return type is Int for signed integers of less than system word size,
 and UInt for unsigned integers of less than system word size. For all 
 other arguments, a common return type is found to which all arguments
 are promoted.

```

---

<div class="post-metadata">

**Author:** ![Mikhail\_Kagalenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikhail_kagalenko/32/13257_2.png) [@Mikhail\_Kagalenko](https://discourse.julialang.org/u/Mikhail_Kagalenko)\
**Post date:** [February 18, 2020, 4:52pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/31 "2020-02-18T16:52:34Z")

</div>

> [@stillyslalom](#):
>
> The behavior of `sum` (and `mean` , by extension)

The assumption that mean() is a mere extension of sum is questionable.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [February 18, 2020, 4:54pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/32 "2020-02-18T16:54:51Z")

</div>

> [@Mikhail\_Kagalenko](#):
>
> If they are interfaced to a computer, all of them output integers, whether you realize it or not. For many examples of thermocouples you consult the [site](https://www.analog.com/en/products/analog-to-digital-converters/integrated-special-purpose-ad-converters/digital-temperature-sensors.html) of one of the largest vendors.

Not really. Most of the devices you are linking work via [I2C](https://en.wikipedia.org/wiki/I%C2%B2C) or [SPI](https://en.wikipedia.org/wiki/Serial_Peripheral_Interface). Strictly speaking, they are just outputting a bunch of bits.

A particular API may represent them as integers for you, but in this case I would argue that it’s just one possible design, and as your example shows, not even a really good one.

Whether you realize it or not 😉, a particularly elegant Julian way of dealing with this would be converting to a temperature representation _at the earliest opportunity_. Then `mean` etc would just work out of the box. See

> **[GitHub - PainterQubits/Unitful.jl: Physical quantities with arbitrary units](https://github.com/PainterQubits/Unitful.jl)**
>
> Physical quantities with arbitrary units. Contribute to PainterQubits/Unitful.jl development by creating an account on GitHub.

eg

```julia
julia> using Unitful, Unitful.DefaultSymbols, Statistics

julia> mean([1u"K", 5u"K"])
3.0 K

```

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [February 18, 2020, 4:58pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/33 "2020-02-18T16:58:16Z")

</div>

It’s right in the [function definition](https://github.com/JuliaLang/Statistics.jl/blob/542f57eabb869099880fdb925c83617199d21bc7/src/Statistics.jl#L170):

```julia
_mean(A::AbstractArray, ::Colon) = sum(A) / length(A)

```

…which is the textbook [definition](https://en.wikipedia.org/wiki/Arithmetic_mean#Definition) of the arithmetic mean

---

<div class="post-metadata">

**Author:** ![Mikhail\_Kagalenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikhail_kagalenko/32/13257_2.png) [@Mikhail\_Kagalenko](https://discourse.julialang.org/u/Mikhail_Kagalenko)\
**Post date:** [February 18, 2020, 5:01pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/34 "2020-02-18T17:01:19Z")

</div>

That is one implementation, and it is flawed. sum() and mean() have different signatures, in Haskell the former would have been

```julia
[a]->a

```

whereas the latter

```julia
[a]->Float

```

Type conversion step is not a “mere extension”

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 18, 2020, 5:04pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/35 "2020-02-18T17:04:37Z")

</div>

That’s not a correct function signature. For example, the mean of complex numbers should be complex. As such, the correct type signature for the return type is the result of dividing the sum of the collection by it’s length.

---

<div class="post-metadata">

**Author:** ![Mikhail\_Kagalenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikhail_kagalenko/32/13257_2.png) [@Mikhail\_Kagalenko](https://discourse.julialang.org/u/Mikhail_Kagalenko)\
**Post date:** [February 18, 2020, 5:06pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/36 "2020-02-18T17:06:11Z")

</div>

> [@Oscar\_Smith](#):
>
> the correct type signature for the return type is the result of dividing the sum of the collection by it’s length.

How do you divide types?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 18, 2020, 5:08pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/37 "2020-02-18T17:08:19Z")

</div>

I don’t mean divide types, I mean assuming that there is a type stable division, the return type of mean should be of that type.

---

<div class="post-metadata">

**Author:** ![Mikhail\_Kagalenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikhail_kagalenko/32/13257_2.png) [@Mikhail\_Kagalenko](https://discourse.julialang.org/u/Mikhail_Kagalenko)\
**Post date:** [February 18, 2020, 5:09pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/38 "2020-02-18T17:09:27Z")

</div>

> [@Oscar\_Smith](#):
>
> assuming that there is a type stable division, the return type of mean should be of that type.

That’s not how Julia’s mean() behaves, though. The type-stable division for integers is div().

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [February 18, 2020, 5:14pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/39 "2020-02-18T17:14:44Z")

</div>

As we’ve discussed at length in this thread, there are pros & cons to widening the accumulator when summing. Type conversion comes from `/(a,b)`, which always returns a float for integer arguments. If we want `mean` to behave differently, we could define `mean(A::AbstractArray{T,N}) where {T <: Integer, N} = sum(A) ÷ length(A)`, but that doesn’t solve your original issue, and introduces new issues:

```julia
julia> mean([1,1]*typemax(Int))
-1

julia> mean([1,2,2])
1

```

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 18, 2020, 5:17pm UTC](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767/40 "2020-02-18T17:17:13Z")

</div>

By type stable division, I don’t mean maintains the same type. I mean has a result that is type stable (ie `Int/Int = Float`) I was being a little pedantic to get around the fact that if the length of your collection is a `BigInt` for example, the result should be a `BigFloat` (or complex or unit…)

[Previous page](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767.md?page=1)

[Next page](https://discourse.julialang.org/t/mean-of-integers-overflows-bug-or-expected-behaviour/34767.md?page=3)