It鈥檚 still the same case from the original PR that brought in DI, since DI broke the test suite 
The problem is DI uses the ForwardDiff.jacobian function which is not GPU compatible. I鈥檝e been saying it should avoid it and use the same trick as SparseDiffTools, or finally patch ForwardDiff.jacobian (though there鈥檚 some trade-offs there).
Issue ForwardDiff scalar indexing with GPU arrays 路 Issue #820 路 JuliaDiff/DifferentiationInterface.jl 路 GitHub, but I haven鈥檛 had a proper GPU on me for a few weeks now so I cannot do the step to make it just the DI call. But it should be pretty clear why it鈥檚 doing it since the ForwardDiff.jacobian call scalar indexes the partials which is exactly the operation that isn鈥檛 allowed and is pointed to in the error message.
You don鈥檛 check for scalar indexing. It鈥檚 slow because of scalar indexing.