Newcomer's guide to Julia package manager

I wrote a blog post to help beginners understand package management in Julia. The intended audience is people who have no familiarity with the concept of packages, versions, etc. I’ve seen a lot of people try to run some Julia code they found somewhere and not understand why it doesn’t just work. This is understandable coming from MATLAB or Stata where this usually isn’t a problem.

To address this, I walk through an example where code fails to run due to some package issue, then suggest how they can solve this by using environments. I recommend using project specific environments for everything, which is admittedly not how I used to do things, but makes sharing and archiving your code much easier and prevents version conflicts.

Comments welcome!

26 Likes

This is great. I hope you work on something similar for package developers and how to use dev in combination with environments. This stuff always gets me confused.

3 Likes

Trying this out with Julia 1.1 and I get this error with using GLM.

julia> using GLM
[ Info: Recompiling stale cache file /Users/peterdeffebach/.julia/compiled/v1.1/GLM/6OREG.ji for GLM [38e38edf-8417-5370-95a0-9cbb8c7f171a]
ERROR: LoadError: Unable to import Arpack_jll on Julia versions older than 1.3!

I think you actually need 1.3 to run this code, not just that it’s recommended. Do you need Julia version bounds in your project.toml?

edit: an ] up in the same environment fixed it. But still less than ideal for a new comer to julia used to just ssc install-ing until the code runs.

I recommend @ChrisRackauckas’s video for package development. He says in the video comments

Note: After "]dev"ing a package, the local version of “using Package” uses the package in your “~/.julia/dev/Package” folder!

I don’t actually know how dev interacts with environments. My guess is that if you run dev in a local project environment, it would update the manifest to point to the dev version, but I’m not sure. It’s a good question.

1 Like

Indeed that is what it does, the Manifest will point to the dev’ed path and it is your responsibility to manage that until you free the package.

1 Like

Good point! I hadn’t realized the latest version of GLM was 1.3 only. The advice still works, but that particular example with that Manifest.toml won’t work.

That being said, I really do recommend 1.3. It’s a lot more pleasant to work with environments in 1.3 because of https://github.com/JuliaLang/julia/pull/32651 which stops repeated precompiles when swapping between environments – at least, that is my understanding of what that PR does.

edit: @pdeffebach I fixed this by creating the Manifest.toml on 1.2 and updating the zip file. It should work on earlier versions now. I tested on 1.1.1 and 1.3.

1 Like

@aaowens Thanks for doing this! I have a noob question, I know you can leave an environment by calling ] activate without the dot at the end. And you can come back to it later. However, when you return to an environment, do you have to do both

] activate .
] instantiate

If not, what do you need instantiate for?

As the documentation says, instantiateis responsible for: " If a Manifest.toml file exist in the current project, download all the packages declared in that manifest. Else, resolve a set of feasible packages from the Project.toml files and install them."

In other words, when a new environment is activated, it needs to be populated by the required packages.
My guess is when switching to the default environment, there is no need for instantiation. I believe it is a no-op in that case.

That’s what I thought. So, you instantiate in case there’s an environment you want to replicate, but if you’re just working on an environment, activate . is enough.

I was not aware that Pkg has now its own website. Is there any other documentation extracted from the Julia docs?

I think it is the other way round: the Pkg docs were not extracted from the manual, a brief summary was incorporated into the manual instead. Which of course links to the actual Pkg manual.

1 Like

The manual in its current form is just the “get started” page of the whole Pkg docs, but in the past it was a very long, exhausting page.

Yes, but that was a different package manager.

Thanks for the post, pkg management has been a bit of a struggle for me.

Would you have something similar for all of us who are working on non-networked systems?

1 Like

I don’t have any experience with that. A Docker image with packages pre-installed would work.