I am trying to save my Flux.jl model and the optimizer to disk. This seems to work just fine when everything is on the CPU but I cannot get the optimizer out of the GPU. cpu(opt) doesn’t seem to do anything. Is this supported? There a dedicated paragraph on saving the state of the optimizer in the documentation [1].
This is one big weak point of the current Flux optimizer design, and there’s no easy way to fix it. That’s why Flux is moving to use GitHub - FluxML/Optimisers.jl: Optimisers.jl defines many standard optimisers and utilities for learning loops., which does let you do cpu(opt_state) and save things that way. If you can, I’d recommend switching to it today so that you’re also future-proofed when Flux switches over.