# Flux relu fails with CUDA gpu

**URL:** <https://discourse.julialang.org/t/flux-relu-fails-with-cuda-gpu/49399>\
**Category:** New to Julia\
**Tags:** flux\
**Created:** [November 1, 2020, 2:53am UTC](https://discourse.julialang.org/t/flux-relu-fails-with-cuda-gpu/49399 "2020-11-01T02:53:39Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![asyrov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/asyrov/32/13578_2.png) [@asyrov](https://discourse.julialang.org/u/asyrov)\
**Post date:** [November 1, 2020, 2:53am UTC](https://discourse.julialang.org/t/flux-relu-fails-with-cuda-gpu/49399/1 "2020-11-01T02:53:39Z")

</div>

I’m getting error “CuArray only supports bits types” when using relu in code like:

```julia
model = Chain(Dense(2, 2, relu), Dense(2, 1, identity)) |> gpu

```

All works fine though with:

```julia
model = Chain(Dense(2, 2, (x) -> max(0, x), Dense(2, 1, identity)) |> gpu

```

Can anyone help how to fix relu or I should use below code?

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [November 1, 2020, 4:39am UTC](https://discourse.julialang.org/t/flux-relu-fails-with-cuda-gpu/49399/2 "2020-11-01T04:39:27Z")

</div>

Can you post the type and size of the input you’re passing to the model? ReLU should work just fine with as long as the inputs are floats:

```julia
julia> x = cu(rand(2, 4))
2×4 CuArray{Float32,2}:
 0.648422 0.723313 0.857359 0.612986
 0.575391 0.343899 0.735907 0.768486

julia> model = Chain(Dense(2, 2, relu), Dense(2, 1, identity)) |> gpu
Chain(Dense(2, 2, relu), Dense(2, 1))

julia> model(x)
1×4 CuArray{Float32,2}:
 -0.0808477 -0.0736467 -0.105517 -0.0689205

```

---

<div class="post-metadata">

**Author:** ![asyrov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/asyrov/32/13578_2.png) [@asyrov](https://discourse.julialang.org/u/asyrov)\
**Post date:** [November 1, 2020, 4:46pm UTC](https://discourse.julialang.org/t/flux-relu-fails-with-cuda-gpu/49399/3 "2020-11-01T16:46:16Z")

</div>

It is ‘Array{Tuple{CuArray{Float32,1},Float64},1}’

Note, it fails in train! method. Here is complete code (it does not work with relu, but if I replace it with ‘(x) → max(o, x)’ it starts to work:

```julia
using Flux
using Flux: throttle
using CUDA

regX = rand(1000)
regY = 50 .+ 1000 * regX + 2 * randn(1000)
regZ = 10 .+ 5 .* regX + 12 .* regY .+ 0.2 * (randn(1000) .- 0.5) 

data = [([regX[i],regY[i]],regZ[i]) for i in 1:length(regX)]
data = gpu.(data)
println(typeof(data))

model = Chain(Dense(2, 2, relu), Dense(2, 1, identity)) |> gpu

loss(x, y) = Flux.mse(model(x), y) 
evalcb = () -> @show(sum([loss(i[1],i[2]) for i in data2D]))

opt = ADAM(0.01)
println(typeof(params(model)))

for i=1:10 
  Flux.train!(loss, params(model), data, opt, cb = throttle(evalcb, 1000)) 
end

```