# Flux: GPU slower than CPU?

**URL:** <https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072>\
**Category:** GPU\
**Tags:** question, performance, flux\
**Created:** [August 9, 2018, 9:25am UTC](https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072 "2018-08-09T09:25:56Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![rkat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rkat/32/3918_2.png) [@rkat](https://discourse.julialang.org/u/rkat)\
**Post date:** [August 9, 2018, 9:25am UTC](https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072/1 "2018-08-09T09:25:56Z")

</div>

I have a loss function which I’ve defined using Flux. On my boring old CPU on one batch, it performs like this:

```julia
julia> @time loss(x, y)
  0.071089 seconds (402 allocations: 20.235 MiB, 4.37% gc time)
1.3092503770133925 (tracked)

```

When running on a machine with a fancy GPU, it performs like this:

```julia
julia> @time loss(x, y)
  1.685316 seconds (1.24 M allocations: 61.493 MiB, 0.59% gc time)
2.6197212f0 (tracked)

```

(and yes, I have run the precompiler before testing in both cases). What’s going on here? Why does the GPU version allocate 3000x the amount of memory and take 25x longer?

---

<div class="post-metadata">

**Author:** ![Roger-luo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roger-luo/32/3399_2.png) [@Roger-luo](https://discourse.julialang.org/u/Roger-luo)\
**Post date:** [August 9, 2018, 9:44am UTC](https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072/2 "2018-08-09T09:44:32Z")

</div>

What is the loss function here?

---

<div class="post-metadata">

**Author:** ![rkat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rkat/32/3918_2.png) [@rkat](https://discourse.julialang.org/u/rkat)\
**Post date:** [August 9, 2018, 9:45am UTC](https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072/3 "2018-08-09T09:45:41Z")

</div>

I’m using the `logitcrossentropy` function that comes with Flux

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [August 9, 2018, 9:58am UTC](https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072/4 "2018-08-09T09:58:43Z")

</div>

Can you post a reproducible example?

---

<div class="post-metadata">

**Author:** ![rkat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rkat/32/3918_2.png) [@rkat](https://discourse.julialang.org/u/rkat)\
**Post date:** [August 9, 2018, 10:07am UTC](https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072/5 "2018-08-09T10:07:19Z")

</div>

Here goes:

```julia
using Flux
# using CuArrays if you are on the GPU
d = Dense(50, 10) |> gpu
n = 50000
x = rand(50, n) |> gpu
hcat([[i == j for i = 1:10] for j = rand(1:10, n)]...) .* 1. |> gpu
loss(a, b) = Flux.logitcrossentropy(d(a), b)

```

CPU:

```julia
julia> @time loss(x, y)
  0.331867 seconds (139.06 k allocations: 77.120 MiB, 35.57% gc time)
2.4550202681078717 (tracked)

```

GPU:

```julia
julia> @time loss(x, y)
 24.348809 seconds (17.95 M allocations: 892.661 MiB, 0.62% gc time)
2.5094972f0 (tracked)

```

[edited to remove the prompt to allow for easier copying]

---

<div class="post-metadata">

**Author:** ![rkat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rkat/32/3918_2.png) [@rkat](https://discourse.julialang.org/u/rkat)\
**Post date:** [August 9, 2018, 10:24am UTC](https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072/6 "2018-08-09T10:24:45Z")

</div>

Also for what it’s worth:

| thing I am using | version of that thing |
| --- | --- |
| julia | 0.6.4 |
| Flux | 0.5.4 |
| CuArrays | 0.6.2 |
| ubuntu | 16.04 |
| gpu | tesla k80 |
| cuda | 9.2 |

---

<div class="post-metadata">

**Author:** ![rkat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rkat/32/3918_2.png) [@rkat](https://discourse.julialang.org/u/rkat)\
**Post date:** [August 10, 2018, 12:38am UTC](https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072/7 "2018-08-10T00:38:09Z")

</div>

Still stumped. Is it an issue with my setup? Can anyone else reproduce?

---

<div class="post-metadata">

**Author:** ![Xiucheng\_Li](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiucheng_li/32/9773_2.png) [@Xiucheng\_Li](https://discourse.julialang.org/u/Xiucheng_Li)\
**Post date:** [August 10, 2018, 5:55am UTC](https://discourse.julialang.org/t/flux-gpu-slower-than-cpu/13072/8 "2018-08-10T05:55:08Z")

</div>

Flux is great but many of its operations have not yet been optimized for GPU. Such as the issue [https://github.com/FluxML/Flux.jl/issues/189](https://github.com/FluxML/Flux.jl/issues/189)