Hi, I am working simultaneously on a few Julia packages. Until now, all these packages are only local, so when I have for instance a local package A depending on another local package B, I add B as a dependency using ]dev B in A’s environment.
But when I do so, the Manifest file of A contains a local link to the folder where B is developed (seems normal). My question is, how do you do when you want to release both A and B publicly somewhere (and still keep working on them locally) ?
The only way I see to get rid of all the local paths in the Manifest file is to add public-url-to-B rather than dev B, but if you do that then you need to commit every single change to B in order to use it (and update the environment), which can be painful for further development and seems unnecessary if you are the only person working on the repo (I guess that for repositories with many contributors, there might not be any other choice). Am I missing something, or is that the price to pay? What is the preferred workflow for such situations?
(I’m assuming you need to commit the Manifest file because your packages are not registered. Is that correct?)
I think the situation is a bit tricky here, because as soon as you want to publicly release both packages, you’ll have to ensure that the publicly available versions are compatible. In other words, you’ll have to avoid situations where the released version of A depends on a not-yet-released (but available on your machine) version of B.
One thing you could do is (let me rename A and B to App and Lib respectively):
commit a clean version of Lib and release it on http://git.server/Lib.jl.git (possibly in a dev branch if you don’t want to advertize it as the new master version)
make App depend on the public release of Lib:
] add http://git.server/Lib.jl.git#dev
commit a clean version of App and publicly release it (again, possibly in a specific branch). While doing so, make sure its Manifest file does not contain any local reference to Lib. This way, anybody cloning or adding App from its public git repository will get a clean dependency on the publicly available version of Lib.
In order to easily develop App, temporarily switch to developing Lib again:
] dev /local/path/to/Lib.jl
do as many local changes as you want in App and Lib, but don’t commit anything at this stage. (otherwise you’d risk having inconsistent commits in your repos)
whenever you get to a working version of both packages and want to commit, go back to step 1.
Indeed, my packages are not registered. I my mind the manifest was always public, but I guess you mean that if App is registered and only depends on registered packages, then the Project file contains all the information needed for installing the dependencies and one can keep the Manifest local (with a developed dependency, and one “just” has to ensure compatibility when releasing public versions) ?
The workflow you propose seems indeed to be a good compromise, I will do that. Switching between the public and local version is still a bit painful, but seems doable if just a few packages are involved (and it can be automated if needed). Thanks.
Exactly. The Manifest.toml is generally made public:
for reproducibility purposes, i.e. when you want users of your package to be able to exactly reproduce your environment, including all direct and indirect dependencies in their specific git revisions. Project.toml only specifies information about direct dependencies, and only cares about compatibility requirements with versions, not specific git revisions.
when some of your dependencies are unregistered packages, and will only be installable if one knows their url. This information is only kept in the Manifest.toml file, not in Project.toml.