# Different results when running on CPU or GPU
**URL:** https://discourse.julialang.org/t/different-results-when-running-on-cpu-or-gpu/42200
**Category:** General Usage
**Tags:** cuda
**Created:** [June 28, 2020, 4:13pm UTC](https://discourse.julialang.org/t/different-results-when-running-on-cpu-or-gpu/42200 "2020-06-28T16:13:05Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@natema](https://discourse.julialang.org/u/natema)
#### Post date: [June 28, 2020, 4:13pm UTC](https://discourse.julialang.org/t/different-results-when-running-on-cpu-or-gpu/42200/1 "2020-06-28T16:13:05Z")
If I do
```julia
using Flux
x = rand(Float32, 10000);
W = rand(Float32, 10000,10000);
y = W*x;
gx = x |> gpu;
gW = W |> gpu;
gy = gW*gx;
(gy[1:10] |> cpu) .- y[1:10]
```
I get something like
```julia
10-element Array{Float32,1}:
-0.0014648438
0.0014648438
-0.0012207031
0.0
0.00024414062
-0.0007324219
-0.0017089844
0.0007324219
0.00024414062
0.0026855469
```
We see then that floating point operations are managed slightly differently on the cpu and gpu. Is it something do to Julia (CuArrays?) or does it boil down to the hardware?
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [June 28, 2020, 5:09pm UTC](https://discourse.julialang.org/t/different-results-when-running-on-cpu-or-gpu/42200/2 "2020-06-28T17:09:17Z")
This is likely just that floating point math isn’t associative, so re-ordering the computations can produce different results. Specifically, GPUs batch operations, which can change the results. CPUs also do because of simd instructions, but they do so differently, so the results can end up different.
---
### Author:  [@ctkelley](https://discourse.julialang.org/u/ctkelley)
#### Post date: [June 28, 2020, 5:26pm UTC](https://discourse.julialang.org/t/different-results-when-running-on-cpu-or-gpu/42200/3 "2020-06-28T17:26:06Z")
The CPU and GPU may also have floating point registers of different sizes. Desktop intel hardware has 80-bit wide registers with guard digits. Server class chips like Xeons and Operons have 128 bit or larger registers and can fit more than one 64bit number in there, but have no guard digits. GPUs also have not guard digits. You should see similar effects if you compare your intel laptop (has guard digits) to a server with Xeons.
---
### Author:  [@Elrod](https://discourse.julialang.org/u/Elrod)
#### Post date: [June 28, 2020, 7:25pm UTC](https://discourse.julialang.org/t/different-results-when-running-on-cpu-or-gpu/42200/5 "2020-06-28T19:25:03Z")
Your typical Intel laptop has 256 bit registers that fit 8 `Float32`. Some recent laptops (Ice Lake) fit 16 `Float32` per register.
80 bit registers are almost never used.
---
### Author:  [@ctkelley](https://discourse.julialang.org/u/ctkelley)
#### Post date: [June 28, 2020, 7:35pm UTC](https://discourse.julialang.org/t/different-results-when-running-on-cpu-or-gpu/42200/6 "2020-06-28T19:35:54Z")
Oh, but they are! Guard digits are used by, for example, any BLAS call. Float64 and Float32 use them routinely. You would not see it in a user code, but when you accumulate sums in a register it happens automatically.
---
### Author:  [@ctkelley](https://discourse.julialang.org/u/ctkelley)
#### Post date: [June 28, 2020, 7:42pm UTC](https://discourse.julialang.org/t/different-results-when-running-on-cpu-or-gpu/42200/7 "2020-06-28T19:42:47Z")
Just looked at the Core I5 hardware specs. @Elrod is right. I’m an old guy and still have 8087s wired into my head.
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [June 28, 2020, 11:15pm UTC](https://discourse.julialang.org/t/different-results-when-running-on-cpu-or-gpu/42200/8 "2020-06-28T23:15:09Z")
One of the easiest ways to figure out if something is going wrong is to go to higher precision.
```julia
julia> using CUDA
julia> let
x = rand(Float32, 10_000)
W = rand(Float32, 10_000, 10_000)
y = W*x
cux = CuArray(x)
cuW = CuArray(W)
cuy = cuW * cux
Array(cuy)[1:10] .- y[1:10]
end
10-element Array{Float32,1}:
-0.00048828125
0.0012207031
0.0007324219
-0.0012207031
0.0021972656
-0.00024414062
-0.0021972656
-0.00024414062
0.0
0.00024414062
```
whereas at 64 bit precision, we see
```julia
julia> let
x = rand(Float64, 10_000)
W = rand(Float64, 10_000, 10_000)
y = W*x
cux = CuArray(x)
cuW = CuArray(W)
cuy = cuW * cux
Array(cuy)[1:10] .- y[1:10]
end
10-element Array{Float64,1}:
1.8189894035458565e-12
-3.183231456205249e-12
7.275957614183426e-12
-1.8189894035458565e-12
-4.092726157978177e-12
-7.73070496506989e-12
4.547473508864641e-13
-2.7284841053187847e-12
1.3642420526593924e-12
9.094947017729282e-13
```
This strongly indicates to me that the problem is just the lack of precision in Float32. Also, you really usually want to know what the _relative_ error is:
```julia
julia> let
x = rand(Float32, 10_000)
W = rand(Float32, 10_000, 10_000)
y = W*x
cux = CuArray(x)
cuW = CuArray(W)
cuy = cuW * cux
(Array(cuy)[1:10] .- y[1:10]) ./ y[1:10]
end
10-element Array{Float32,1}:
-1.9602805f-7
4.906113f-7
-3.875746f-7
-1.9639621f-7
-1.0653941f-6
2.9447702f-7
-2.9167163f-7
4.891585f-7
-9.862888f-8
-1.9692011f-7
```
So while the `Float32` calculation had what looked like large differences between the GPU and CPU result, relative to the magnitude of that result, the differences are actually quite small.