That ifelse looks somewhat suspect, try extracting it as an element-wise function:
weirdrelu(x) = ifelse(x > 0, x, one(x))
f(X) = sum(weirdrelu.(X))
gradient(f, X)
Returns ([1, 1, 1],) as expected on the latest CUDA.jl.
That ifelse looks somewhat suspect, try extracting it as an element-wise function:
weirdrelu(x) = ifelse(x > 0, x, one(x))
f(X) = sum(weirdrelu.(X))
gradient(f, X)
Returns ([1, 1, 1],) as expected on the latest CUDA.jl.