How to properly update Julia without re-uploading packages

Hello,
I have downloaded the new Julia (1.9) – well done to the developers, by the way.
I don’t know if there is a straightforward way to upgrade julia core (in R, one can type in the terminal $ sudo add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/'; sudo apt install r-base), so I simply replace the julia’s folders with each release.
The problem is that each time I upgrade, I need to reinstall all packages. In fact, for instance, I get the error:

julia> using DifferentialEquations
ERROR: ArgumentError: Package DifferentialEquations not found in current path.
- Run `import Pkg; Pkg.add("DifferentialEquations")` to install the DifferentialEquations package.

Since the error here looks to be a path, is it possible to set a common path so that I don’t have to reinstall all packages each time?
COpying the new installations in my julia folder is straightforward, reinstalling the packages is time-consuming.
Thank you

You are not supposed to remove the ~/.julia folder upon upgrading Julia, because you will also remove all of your environments, (~/.julia/environments) which hold your downloaded packages, too.

You should take a look at this GitHub - JuliaLang/juliaup: Julia installer and version multiplexer for managing versions with Julia.

IIRC you really can’t get around on re-instantiating (and precompiling) packages if you upgrade your julia version.

And if you suffer from long upgrade times due to having lots of packages in your global environment, it is recommended to migrate your workflow to use separate environments with only the relevant packages.

2 Likes

Your issue is likely that you have installed all packages into the global environment, which is Julia version specific. If you followed the recommended workflow of having project-specific environments, you could just activate those environments in the new Julia version and it would (probably) work (subject to the environment getting re-resolved as not all packages might support both Julia versions, or at least not at the same package version).

If you just want to move your default environment, you can look in /.julia/environments/vX.X and copy the project file over to /.julia/environments/v1.9

3 Likes

If I get it right, I should remove my current julia folder (I keep it in ~/src/julia) and use juliaup; in that case, where is the julia exe file stored?
Alternatively, I can keep my current ~/src/julia, copy the new versions in it, but keep a copy of /.julia/environments so as not to re-install all packages each time?

If I get it right, I should remove my current julia folder (I keep it in ~/src/julia) and use juliaup; in that case, where is the julia exe file stored?

No, juliaup generates a folder ~/.julia/juliaup in which it installs and manages everything.
Inside ~/.julia/juliaup/bin you can then find a symlink called julia which is what will be managed by juliaup when swapping out versions. This is the directory you should add to your PATH then, because it also contains the juliaup exe.

If you follow the instructions on their github page and download the installer, it should walk you through a setup where you can point it to an already exisiting julia folder and things should get set up accordingly (modulo the PATH variable, I think).

1 Like

The question of how to update Julia and where to put your Julia installations is completely unrelated to the question of why using DifferentialEquations does not work when it is not installed in the environment which is currently active.

The only reason things might have become a bit muddled here is that each Julia version has a default environment (which takes its name from the version, e.g. v1.9 for Julia 1.9). Still these aren’t actually tied to a version (apart from the fact that they are used by default), consider:

julia> versioninfo()
Julia Version 1.9.0
(...)

(@v1.9) pkg> activate .../.julia/environments/v1.4
  Activating project at `...\.julia\environments\v1.4`

(@v1.4) pkg> st
Status `...\.julia\environments\v1.4\Project.toml`
⌃ [336ed68f] CSV v0.10.4
⌃ [a93c6f00] DataFrames v1.3.6
  [7073ff75] IJulia v1.24.0
Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> using DataFrames
[ Info: Precompiling DataFrames [a93c6f00-e57d-5684-b7b6-d8193f3e46c0]

Here I am using Julia 1.9, but am activating the 1.4 environment and the use the (outdated) version of DataFrames which was recorded in that environment.

2 Likes

Can also be written as pkg> activate --shared v1.4.

2 Likes

Ah yes, and even shorter as activate @v1.4

1 Like

this was just an example: all packages give the same error

Yes of course, and that is expected as your default environment for any new Julia version will be empty when first used.

You might want to read through

https://pkgdocs.julialang.org/v1/environments/

to get a better understanding of what environments are and how they are used. I know you are a scientist so are presumably using Julia to perform analyses which ends up in published papers and should ideally be reproducible, which is another important reason to work with project-specific environments.

Hi @Luigi_Marongiu,

maybe this helps:

Can we maybe prompt user asking if they wanna migrate Project.toml from previous version of Julia?

Yes/No/Later

1 Like

Can we maybe prompt user asking if they wanna migrate Project.toml from previous version of Julia?

Yes, I’ve been thinking a little bit how that should be done best. Perhaps, at REPL init time, check:

  • We are running a julia version without a `.julia/environments/v1.x/Project.toml
  • There exists a `.julia/environments/v1.(x-1)/Project.toml
  • Write an info message in the REPL saying that "to transition packages from the previous julia version, run Base.transition_package()

Or something like that. Having it be interactive sounds like it could get annoying.

5 Likes

I installed juliaup, it looks like it is working good so far. Thank you

2 Likes

I have a conda environment where I have installed Julia. I want to update Julia in this environment.

  • Using juliaup, how do I choose a folder in which to install Julia?
  • How do I uninstall Julia?
  • How do I uninstall juliaup?

It would be nice if these features were built in.

If I shouldn’t be doing this in conda environments, please let me know.

I don’t understand environments very well. I did read through some of the documentation, but I have more work to do there. I thought all environments were “project environments”. But turns out there are “package directories” and “stacked environments” too. Any quick tips or reading geared towards non-computer-scientists are welcome.

Using juliaup, how do I choose a folder in which to install Julia?

Which OS are you on?

If on Linux, you run the command from the github page and it prompts you into a dialog asking for an installation path for the folder. I think this is similar on MacOS, but I don’t know how it works with the Windows Store option.

How do I uninstall Julia?
How do I uninstall juliaup?

On Linux (and I think MacOS) you just delete the juliaup folder into which you installed in the first place. This removes any julia versions and also juliaup.
How to do this on Windows from the Store is still being discussed, see Uninstalling Juliaup doesn't remove .julia dir or at least .julia/juliaup · Issue #538 · JuliaLang/juliaup · GitHub

If I shouldn’t be doing this in conda environments, please let me know.

Don’t use conda.

I don’t understand environments very well. I did read through some of the documentation, but I have more work to do there. I thought all environments were “project environments”. But turns out there are “package directories” and “stacked environments” too. Any quick tips or reading geared towards non-computer-scientists are welcome.

This is a more applied guide on how to work with environments:

You might want to read through
https://pkgdocs.julialang.org/v1/environments/

Thanks @fatteneder . I’m on MacOS right now. I don’t recall getting the option to choose path, but I did this some time back so don’t remember very well. I think there were 3 options and I chose the “standard” installation. Maybe the “advanced” option would have let me choose a folder.