Taking zygote gradient on function with ! operator fails on gpu

Specific Domains Machine Learning
,
1

ERROR: LoadError: MethodError: no method matching !(::ForwardDiff.Dual{Nothing,Bool,1})

Only happens on gpu, not cpu, which is really confusing me.
MWE
using Flux
masksum(x,y) = sum(.!x * y)
gradient(masksum, [true,false,true] |> gpu, .3)

I would appreciate any assistance. I don’t understand why putting it on the gpu would make it fail, and the error message doesn’t really illuminate.

2

Differentiating through boolean does really make sense as your can’t change a bool a little bit. The below works.

using Flux
masksum(x,y) = sum(1 .- x * y)

gradient(masksum, [1,0,1], .3)

gradient(masksum, [1,0,1] |> gpu, .3)
3 Likes
3

True, I’m not using the gradient with respect to the mask, but this was the simplest way to demonstrate.

Thanks! got too into debugging why it was working on cpu but not gpu and stopped looking for workarounds after not(x) = x ? false : true crashed cuda.

4

on cpu it returned nothing so… maybe gpu didn’t have logic to deal with nothing.

1 Like