I am developing package P with dependency D on a local computer. Both packages are unregistered.
Locally, changes are tracked by using Pkg.develop(D) and Revise.
The actual computations are run on a remote machine.
The code is uploaded with rsync.
Problem 1: Ps Manifest.toml points to a directory for Ds location that does not exist on the remote machine.
I can solve this by issuing Pkg.add(D) before the rsync. But that leads to
Problem 2: How do I force the remote machine to recompile D using the latest version of the code (either in .julia/dev or on github)?
As far as I can tell, Pkg.update(D) does nothing because D is not registered.
I feel like the obvious solution would be to mirror the directory structure on both machines, so that you not only rsync the program over, but you rsync all the dependent packages as well.
If you don’t want to use your development structure on the other machine it looks like you could use the JULIA_PKG_DEVDIR environment variable and then just mirror the dependent packages to the JULIA_PKG_DEVDIR on the remote machine(s). The documentation says you would need to do Pkg.develop() by their URL instead of path in that case.
I’m not sure what you mean by the Pkg.add(D) before the rsync…
Thank you those suggestions.
I cannot mirror full paths on the remote machine, and Manifest contains full paths.
I do set up the directories such that the paths on both machines are the same relative to JULIA_PKG_DEVDIR.
You suggest to do Pkg.develop with a github url. I believe the remote machine will download from github once and then only track changes in the remote machine’s JULIA_PKG_DEVDIR. The entry in Manifest.toml is the same as with Pkg.develop(path to pkg).
What I meant by Pkg.add before rsync is: avoid ending up with paths in Manifest that are not valid on the remote machine by replacing them with urls.
Thanks again for your help.
If D lives on github and you Pkg.add("github URL of D") to a project than you can use Pkg.instantiate and, later, Pkg.update on the remote machine and it should work as for registered packages (modulo versioning).
Thank you - I just tested this suggestion.
So far, this works after restarting the REPL. Otherwise, Julia does not recompile and therefore keeps working with an outdated version of D.
Given the way I use the remote machine, this may be a solution.
I am just not sure yet that recompiling happens every time after restarting the REPL. I have never been able to figure out how to trigger a recompile.