When you want to modify your new package to add a dependency. You should do the following.
julia> using Pkg
julia> Pkg.activate("/Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM")
julia> Pkg.add("NameOfDependency")
julia> Pkg.activate() # return to default enviornment, or better yet, select a more specific enviornment.
Yes. This is the point. It uses the location we pointed develop to as the package store for that package. Whatever is located at that file system location is what is used when you do using CovidSim_ilm for the first time in a Julia session.
Tables.jl was not in “/Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM/Project.toml” when you tried to invoke using CovidSim_ilm and/or you had already deleted Tables.jl from ~/.julia/packages.
We methodically started with your package environment, a temporary enviornment, and then your default enviornment. At each step we instantiated the enviornment to ensure it was in a consistent state, then we made sure to add or develop the necessary packages to each environment. We did not delete stuff in various places causing the system to get into an inconsistent state.
OK. I have to re-read Pkg.activate(). I never understood it.
I may have mislead you: I never deleted Tables at all (not with rm and not manually). I got rid of the package directory of my own package. I knew that develop should mean the package store was at the local directory and not in my installed packages (based on the git repo), but I incorrectly assumed it also resided in packages, but was not being loaded. But, that was a vestige of previously having added it (though I subsequently rm’ed it–but according to Pkg rm makes it inactive, but still there to permit fast reinstallation).
So, have to do instantiate to select the environment to make changes with Pkg. Have to switch back to default environment after that (optionally).
I am not thrilled with environments with version specific dependencies–that was the minor disaster that Python caused for itself. I prefer using latest stable packages everywhere in Julia and avoiding differential environments. But, that can’t be the case for a “develop’ed” package, which is by definition its own environment.
Another thing that Julia does better than Python (in this packages and environments realm–there are lots of other better things) is even when you set up a project specific environment, the packages are still kept in the user’s centralized package repository–just that there can be multiple versions of packages and the manifest can identify a package by version.
Finally, it adds a lot of complexity to combine the developer focused features of Pkg–into which I made my first foray–and the user focused features (for simply loading and accessing packages). Developers need the full suite of features so putting them in separate packages would be a serious inconvenience but us dumb users want to ignore the extra complexity.
Thank you again for doing this live debugging session. That was helpful beyond any reasonable level of duty!
It basically selects the active Project.toml and Manifest.toml pair.
Actually, no. Only activation is usually necessary. Pkg.instantiate() is only needed when your Julia depot does not contain packages described in the Project.toml and Manifest.toml. The only time I need this is when I manually copy or clone a Julia project to a new computer or if I’m using a new Julia depot. Usually, Pkg.develop or Pkg.add will take care of getting package dependencies installed.
In summary, Pkg.instantiate() allows you to recreate an environment from the Project.toml and Manifest.toml files. In our cass, it also helped to ensure a consistent environment state.
You actually have not declared any version specific dependencies. Your Project.toml lacks a compat section.
See 6. Compatibility · Pkg.jl for more information about adding compatibility bounds for your dependencies.
What I guess you have drawn this conclusion from is the Manifest.toml. The Manifest.toml merely describes the versions and/or locations of all packages in the environment. This allows you to reproduce the environment at a later time or on another computer.
Scientifically, this is also critical so one can reproduce results.
Most library packages do not include a Manifest.toml under version control. This means that thrme latest dependencies will be installed as long as they also satisfy the compat bounds.
Pkg.develop actually ignores the package Manifest.toml when doing package resolution for the environment. Pkg.add does use the Manifest.toml if it is checked into git.
As I demonstrated with my minimal example, you usually do not need to touch Pkg.instantiate or the TOMLs directly. Usually, only Pkg.add and Pkg.develop are necessary.
To be honest, I think Julia’s packaging and environments can be confusing especially if we try to map concepts from our experience with Python on it.
I am working on my first package ever and I was able to relate to the OP’s confusions. Here’s one thing that possibly got left out, which might be helpful for other new package developers in case they come across this thread:
Suppose you have a local package called MyPackage and you want to load it (that is, do using MyPackage) from your default environment. (As an aside, when I say being in your local environment, I mean you see something like (@v1.9) pkg> when you go into the pkg REPL by doing ]. Or by doiong Pkg.activate())
You would dev into your package from your default environment, if that’s the right way to say it-- meaning from your default environment, you do Pkg.develop(path="path/to/MyPackage"). This “adds” MyPackage to your default environment (that is, MyProject appears in the default environment’s Project.toml).
Say you now update MyPackage with a new dependency. You will probably want to do a resolve() from within MyPackage’s environment, or instantiate() if the dependency is not already installed.
But you will also want to do a resolve() from your default environment so that the default environment knows that MyPackage has that additional dependency.