Troubleshooting compatibility issues (preventing RL from downgrading Metal to 0.2)

Tooling Package Management
10

I have fallen and I can’t get up! Please help!

ReinforcementLearning (RL) has switched to sub-packages under the src directory. Older versions of these same sub-packages are available as packages in the Julia general registry.

I checked out a local copy of RL for development

(@v1.10) pkg> st
Status `~/.julia/environments/v1.10/Project.toml`
⌃ [b2108857] Lux v0.5.15
⌃ [dde4c033] Metal v0.5.0
  [158674fc] ReinforcementLearning v0.11.0-dev `~/Work/Julia/ReinforcementLearning.jl`

and activated my local copy

(@v1.10) pkg> activate ~/Work/Julia/ReinforcementLearning.jl
  Activating project at `~/Work/Julia/ReinforcementLearning.jl`

I don’t know why the top RL Project.toml does not specify compatibility with the latest versions of its sub-packages. Perhaps, it doesn’t matter.

(ReinforcementLearning) pkg> st
Project ReinforcementLearning v0.11.0-dev
Status `~/Work/Julia/ReinforcementLearning.jl/Project.toml`
   [189a3867] Reexport v1.2.2
→⌅ [e575027e] ReinforcementLearningBase v0.11.1
→⌅ [de1b191a] ReinforcementLearningCore v0.9.4
→⌃ [dd1544ca] ReinforcementLearningDatasets v0.1.0
→⌃ [25e41dd2] ReinforcementLearningEnvironments v0.7.2
→⌅ [d607f57d] ReinforcementLearningZoo v0.6.0

The sub-packages do require the latest versions of their sibling packages so I need to dev the sibling package that each sub-package requires. There’s a script (devmode.jl, top level in RL) that simplifies the setup, e.g.

function activate_devmode!()
    @info "Switching to dev mode. You are now using your local versions of the RL.jl packages instead of the registered releases."
    #RLBase
    #No dependency to dev
    #RLCore
    Pkg.activate("src/ReinforcementLearningCore")
    Pkg.develop(path="src/ReinforcementLearningBase")
    #RLZoo
    Pkg.activate("src/ReinforcementLearningZoo")
    Pkg.develop(path="src/ReinforcementLearningCore")
    Pkg.develop(path="src/ReinforcementLearningBase")    
    #RLEnvironments
    Pkg.activate("src/ReinforcementLearningEnvironments")
    Pkg.develop(path="src/ReinforcementLearningBase")
    #RLExperiments
    Pkg.activate("src/ReinforcementLearningExperiments")
    Pkg.develop(path="src/ReinforcementLearningZoo")
    Pkg.develop(path="src/ReinforcementLearningEnvironments")
    Pkg.develop(path="src/ReinforcementLearningCore")
    Pkg.develop(path="src/ReinforcementLearningBase")
    #RL
    Pkg.activate(".")
    Pkg.develop(path="src/ReinforcementLearningZoo")
    Pkg.develop(path="src/ReinforcementLearningEnvironments")
    Pkg.develop(path="src/ReinforcementLearningCore")
    Pkg.develop(path="src/ReinforcementLearningBase")
end

It works until it gets to src/ReinforcementLearningExperiments and then

(ReinforcementLearningExperi...) pkg> dev src/ReinforcementLearningZoo
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package ReinforcementLearningCore [de1b191a]:
 ReinforcementLearningCore [de1b191a] log:
 ├─possible versions are: 0.1.0-0.13.0 or uninstalled
 └─restricted to versions 0.15 by ReinforcementLearningExperiments [6bd458e5] — no versions left
   └─ReinforcementLearningExperiments [6bd458e5] log:
     ├─possible versions are: 0.5.0 or uninstalled
     └─ReinforcementLearningExperiments [6bd458e5] is fixed to version 0.5.0

The ReinforcementLearningCore sub-package is at version 0.15 whereas the one available in the Julia registry is at 0.13. The ReinforcementLearningExperiments sub-package is expecting the latest.

Maybe I can dev the ReinforcementLearningCore sub-package to resolve the above error?

(ReinforcementLearningExperi...) pkg> dev src/ReinforcementLearningCore
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package ReinforcementLearningZoo [d607f57d]:
 ReinforcementLearningZoo [d607f57d] log:
 ├─possible versions are: 0.1.0-0.8.3 or uninstalled
 └─restricted to versions 0.10.1-0.10 by ReinforcementLearningExperiments [6bd458e5] — no versions left
   └─ReinforcementLearningExperiments [6bd458e5] log:
     ├─possible versions are: 0.5.0 or uninstalled
     └─ReinforcementLearningExperiments [6bd458e5] is fixed to version 0.5.0

The Zoo sub-package is at version 0.10 but the one in the Julia registry is at 0.8.3.

So old packages in the Julia registry interfere with the ones I’m trying to develop. How do I work around this problem?

Thank you, Joel