This is probably not what you are looking for but in principle you could split up your code into three parts: general, GPU-specific, and CPU-specific.
Then users could do something like:
using GeneralPackage, GPUPackage
result = GeneralPackage.calculate(inputs, GPUPackage.Engine())
or conversely
using GeneralPackage, CPUPackage
result = GeneralPackage.calculate(inputs, CPUPackage.Engine())