# Normalizing a vector to sum to one does not work?

**URL:** <https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560>\
**Category:** General Usage\
**Tags:** numbers, vector\
**Created:** [October 11, 2021, 11:27am UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560 "2021-10-11T11:27:13Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dan.phi](https://avatars.discourse-cdn.com/v4/letter/d/c89c15/32.png) [@Dan.phi](https://discourse.julialang.org/u/Dan.phi)\
**Post date:** [October 11, 2021, 11:27am UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/1 "2021-10-11T11:27:13Z")

</div>

This is an extremely basic question and I have to be missing something, but when trying to normalize matrix A below so that rows sum to one, some (small) differences remain:

```julia
A = rand(100,4)

A = A./sum(A, dims = 2)

sum(sum(A, dims =2) .== 1) # fewer than 100!

```

Why does this happen? Is there a way to solve this?

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [October 11, 2021, 11:35am UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/2 "2021-10-11T11:35:38Z")

</div>

The result is due to floating point inaccuracies. The following holds:

```julia
A = rand(100,4)

A = A./sum(A, dims = 2)

@assert sum(sum(A, dims =2) .≈ 1) == 100

```

Alternate name for `≈` is `isapprox`.

---

<div class="post-metadata">

**Author:** ![Dan.phi](https://avatars.discourse-cdn.com/v4/letter/d/c89c15/32.png) [@Dan.phi](https://discourse.julialang.org/u/Dan.phi)\
**Post date:** [October 11, 2021, 11:42am UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/3 "2021-10-11T11:42:13Z")

</div>

Yes, but I need exact sums–I’m thinking of columns as probabilities.

---

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [October 11, 2021, 11:51am UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/4 "2021-10-11T11:51:43Z")

</div>

Almost all scientific computing code works with inexact probabilities since the standard level of inexactness offered by floating point is usually not an issue in practice. What level of inexactness creates problems for you?

---

<div class="post-metadata">

**Author:** ![Dan.phi](https://avatars.discourse-cdn.com/v4/letter/d/c89c15/32.png) [@Dan.phi](https://discourse.julialang.org/u/Dan.phi)\
**Post date:** [October 11, 2021, 12:00pm UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/5 "2021-10-11T12:00:32Z")

</div>

Well, that eventually results in some distance measures between probability distributions being negative. But now that I know the reason, I will just force them ex-post to be zero.

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 11, 2021, 12:18pm UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/6 "2021-10-11T12:18:07Z")

</div>

As @johnmyleswhite mentions with `Float` types we should always expect these tiny inaccuracies that we force into [0,1] when dealing with probabilities.

However, if you need to guarantee calculations without forcing the outcome an option is to use Rational numbers to declare probabilities.

```julia
A = rand(1:10_000_000,100,4)
A = A .// sum(A, dims = 2)

sum(sum(A, dims =2) .== 1)
100

```

---

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [October 11, 2021, 12:33pm UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/7 "2021-10-11T12:33:19Z")

</div>

The big limitation here is the rational numbers overflow in ways potentially more surprising than floating point numbers:

```julia
julia> x = 1//10
1//10

julia> typeof(x)
Rational{Int64}

julia> for i in 1:1_000
           x *= 1//10
           println((i, x))
           end
(1, 1//100)
(2, 1//1000)
(3, 1//10000)
(4, 1//100000)
(5, 1//1000000)
(6, 1//10000000)
(7, 1//100000000)
(8, 1//1000000000)
(9, 1//10000000000)
(10, 1//100000000000)
(11, 1//1000000000000)
(12, 1//10000000000000)
(13, 1//100000000000000)
(14, 1//1000000000000000)
(15, 1//10000000000000000)
(16, 1//100000000000000000)
(17, 1//1000000000000000000)
ERROR: OverflowError: 1000000000000000000 * 10 overflowed for type Int64
Stacktrace:
 [1] throw_overflowerr_binaryop(op::Symbol, x::Int64, y::Int64)
   @ Base.Checked ./checked.jl:154
 [2] checked_mul
   @ ./checked.jl:288 [inlined]
 [3] *(x::Rational{Int64}, y::Rational{Int64})
   @ Base ./rational.jl:334
 [4] top-level scope
   @ ./REPL[3]:2

```

This is actually part of the general problem – given a fixed number of bits, you can trade off exactness for range or range for exactness. Rational numbers have less range, but are exact; floating point have much larger range but are not exact.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [October 11, 2021, 12:41pm UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/8 "2021-10-11T12:41:26Z")

</div>

I think this is relevant here:

> [@PSA: floating-point arithmetic](https://discourse.julialang.org/t/psa-floating-point-arithmetic/8678):
>
> Sometimes people are surprised by the results of floating-point calculations such as julia\> 5/6 0.8333333333333334 # shouldn't the last digit be 3? julia\> 2.6 - 0.7 - 1.9 2.220446049250313e-16 # shouldn't the answer be 0? These are not bugs in Julia. They’re consequences of the IEEE-standard 64-bit binary representation of floating-point numbers that is burned into computer hardware, which Julia and many other languages use by default. Brief explanation You can t…

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [October 12, 2021, 2:13pm UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/9 "2021-10-12T14:13:57Z")

</div>

24 posts were split to a new topic: [Discussion about integer overflow](https://discourse.julialang.org/t/discussion-about-integer-overflow/69627)

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [October 12, 2021, 2:18pm UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/10 "2021-10-12T14:18:28Z")

</div>

A post was merged into an existing topic: [Discussion about integer overflow](https://discourse.julialang.org/t/discussion-about-integer-overflow/69627/25)

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [October 12, 2021, 2:42pm UTC](https://discourse.julialang.org/t/normalizing-a-vector-to-sum-to-one-does-not-work/69560/11 "2021-10-12T14:42:10Z")

</div>

A post was merged into an existing topic: [Discussion about integer overflow](https://discourse.julialang.org/t/discussion-about-integer-overflow/69627/30)