I have developed a package MyPackage that has it’s own Project.toml and Manifest.toml files. I’ve dev this from my main environment so that my v1.2 Project.toml file looks like this:
(v1.2) pkg> st
Status `~/.julia/environments/v1.2/Project.toml`
[3c78c848] MyPackage v0.1.0 [`~/.julia/dev/mmrvaccinedelay`]
This allows me to do using MyPackage and everything works beautifully. MyPackage has the following code in it
module MyPackage
using DataFrames
using CSV
# module code
end
Both DataFrames and CSV exists in the project’s Project.toml file (but they also exist in the v1.2 Project.toml file). The question is, which one does my package look at. i.e. when I type in using MyPackage from the main v1.2 environment, does the package use DataFrames, CSV from the v1.2 Project.toml or the package’s own Project.toml.
So what’s the purpose of the project having it’s own environment files? How do I force it so that when someone runs using MyPackage, all the dependencies of MyPackage are pulled from the Project.toml file instead of the main environment (or whatever other environment) the use user is in.
If I am understanding this correctly, whenever a using xxx statement is run, it will look for the package based on the Project.toml of the current user environment?
So when someone uses my package (add, using MyPackage), the user has to be in the MyPackage env for the correct packages to be pulled? For example, the DataFrames package version used in my MyPackage Project.toml file is slightly different than what most people would have in their updated 1.3 environment.
How can I ensure that Pkg pulls the older version that my package requires, without the user manually switching environments?
When you run Pkg> add
then the package manager determines which version of your package and of its dependencies to add to the enviroment such that none of the the compat bounds of the users current enviroment or any package in there are violated.
Generally one of the advantages of not working in the global enviroment is that you don;t have a million different packages with a million different bounds all fighting about what shold be used.
Thats why I advise everyone to have 1 enviroment per conceptual project.
And to keep the global enviroment just for a few very commonly used tools.
Something isn’t making sense to me for some reason.
It seems that the ideal workflow for anyone using my package is to create a new environment, and then do add MyPackage, using MyPackage from that environment which will pull dependencies based on the project’s Project.toml file.
What happens when the user does add MyPackage, using MyPackage from their v1.3 environment, which is likely the case with many users? How does Pkg determine what versions of the dependencies to download? For example, MyPackage has using DataFrames and a particular version of DataFrames in the Project.toml file?
Which version of DataFrames is downloaded and used within the package? Is it the version that’s inside the project’s Project.toml file? Or is it the version that’s inside the active environment Project.toml (according to @fredrikekre, the answer is active, in this case v1.3).
Follow up if the answer above is “active”: Suppose the user dosn’t have DataFrames in their active environment. Since my MyPackage is calling using DataFrames, how will Pkg know which version of DataFrames to download?
No, there are no particular versions of the dependencies in the Project.toml file. Particular versions can be found in the Manifest.toml file.
The general idea is that Project.toml lists the dependencies and compatibility bounds (compat section) for which versions will work with your package. When someone adds your package to one of their environments, it’s the responsibility of the Pkg resolver to find dependency versions that work with all the packages in the environment and download those. If the packages in the environment have contradictory compatibility requirements it will refuse to add your package.
Yes and no. It will pull the list of dependencies from your Project.toml file, and in the absence of other packages with different requirements will resolve the versions to the highest available versions (*) that are consistent with your compat bounds.
In order to replicate exactly the same environment, what you should do is to commit both Project.toml and Manifest.toml, then instruct the user to activate your package (Pkg.activate or start with --project) and Pkg.instantiate to download the specific dependency versions found in Manifest.toml.
(*) I hope I’m right on that detail. Otherwise someone will have to correct me.
Thank you. This was the explanation that really tied it all together for me. Now I know how to package my scientific projects as well as use DrWatson.jl to manage everything.