JuMP installation issue in Julia v1.6

erans · April 6, 2021, 11:03pm

Hello,

Something rather strange just happened when I tried to use JuMP with Julia v1.6 and I’m curious if it’s a bug, or if I can get some explanation on why something like this would happen.

(I hope this is the appropriate place to pose the questions, if not, please direct me to the appropriate channel)

Step 1 (in Julia v1.6)

I start a brand new project (tmp) and add JuMP, only to have some of the dependencies error:

(@v1.6) pkg> activate .
  Activating new environment at `C:\Users\erans\Documents\tmp\Project.toml`

(tmp) pkg> status
      Status `C:\Users\erans\Documents\tmp\Project.toml` (empty project)

(tmp) pkg> add JuMP
    Updating registry at `C:\Users\erans\.julia\registries\General`
    Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Resolving package versions...
    Updating `C:\Users\erans\Documents\tmp\Project.toml`
  [4076af6c] + JuMP v0.21.6
    Updating `C:\Users\erans\Documents\tmp\Manifest.toml`
  [6e4b80f9] + BenchmarkTools v0.5.0
  [49dc2e85] + Calculus v0.5.1
  [d360d2e6] + ChainRulesCore v0.9.37
[...]
 [3f19e933] + p7zip_jll
Precompiling project...
  Progress [========================================>]  5/5
  ✗ CompilerSupportLibraries_jll
  ✗ MbedTLS_jll
  ✗ Zlib_jll
  ✗ LibSSH2_jll
  ✗ ZipFile
0 dependencies successfully precompiled in 10 seconds (30 already precompiled)
5 dependencies errored

julia> using JuMP
[ Info: Precompiling JuMP [4076af6c-e467-56ae-b986-b466b2749572]
ERROR: LoadError: ArgumentError: Package Zlib_jll does not have JLLWrappers in its dependencies:
- If you have Zlib_jll checked out for development and have
  added JLLWrappers as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with Zlib_jll
[...]
ERROR: LoadError: Failed to precompile Zlib_jll [83775a58-1f1d-513f-b197-d71354ab007a] to C:\Users\erans\.julia\compiled\v1.6\Zlib_jll\jl_B090.tmp.
[...]
ERROR: LoadError: LoadError: Failed to precompile CodecZlib [944b1d66-785c-5afd-91f1-9de20f533193] to C:\Users\erans\.julia\compiled\v1.6\CodecZlib\jl_ADD1.tmp.
[...]
ERROR: LoadError: Failed to precompile MathOptInterface [b8f27783-ece8-5eb3-8dc8-9495eed66fee] to C:\Users\erans\.julia\compiled\v1.6\MathOptInterface\jl_7319.tmp.
[...]
ERROR: Failed to precompile JuMP [4076af6c-e467-56ae-b986-b466b2749572] to C:\Users\erans\.julia\compiled\v1.6\JuMP\jl_6E5C.tmp.
[...]

Step 2 (In Julia v1.5)

So, I delete the Project.toml and Manifest.toml files and repeat the same exact thing but now in a Julia v1.5.3 REPL:

(@v1.5) pkg> activate .
 Activating new environment at `C:\Users\erans\Documents\tmp\Project.toml`

(tmp) pkg> status
Status `C:\Users\erans\Documents\tmp\Project.toml` (empty project)

(tmp) pkg> add JuMP
   Updating registry at `C:\Users\erans\.julia\registries\General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
Updating `C:\Users\erans\Documents\tmp\Project.toml`
  [4076af6c] + JuMP v0.21.6
Updating `C:\Users\erans\Documents\tmp\Manifest.toml`
  [56f22d72] + Artifacts v1.3.0
  [6e4b80f9] + BenchmarkTools v0.5.0
  [6e34b625] + Bzip2_jll v1.0.6+5
  [49dc2e85] + Calculus v0.5.1
  [...]
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

julia> using JuMP

julia>

No errors

Step 3 (In Julia v1.6…again)

Finally the part that I really don’t understand.
I now open the Julia v1.6 REPL again, navigate to this project folder and activate it.

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)

julia> cd("Documents\\tmp\\")

(@v1.6) pkg> activate .
  Activating environment at `C:\Users\erans\Documents\tmp\Project.toml`

(tmp) pkg> status
      Status `C:\Users\erans\Documents\tmp\Project.toml`
  [4076af6c] JuMP v0.21.6

julia> using JuMP

julia> model = Model()
A JuMP Model
Feasibility problem with:
Variables: 0
Model mode: AUTOMATIC
CachingOptimizer state: NO_OPTIMIZER
Solver name: No optimizer attached.

Huh??

Summary

I can’t seem to add JuMP in version 1.6
I can add JuMP in version 1.5
After adding JuMP in version 1.5 I can use it in version 1.6

This does not seem like it’s how things should work.
Your help is greatly appreciated.

Henrique_Becker · April 7, 2021, 1:09am

If you have the Julia 1.5 and Julia 1.6 binaries and you just alternate between calling one and the other then both are writing to the same Julia folder in your PC (where the packages are saved together with the REPL history and things like that). So it is expected that they may affect each other, but I am not sure to what extent, and if it is entirely safe to just alternate between binaries like that. I have always kept a different Julia folder for each version and before I start calling a version I often do not use I move my current ~/.julia to ~/.julia-OLD-VERSION-NUMBER and either let the binary create a new folder or move a folder for that version I saved back to that place.

odow · April 7, 2021, 1:22am

What’s the version of Zlib_jll showing when you add JuMP the first time in 1.6?

Are you on a work/university network? Julia 1.6 changed how things are downloaded, and there may be some issues with firewalls, etc.

erans · April 7, 2021, 1:38am

Hmmm…I would have thought the binaries address handled similarly to the packages themselves in the manifest, so that hypothetically I could have two different binary versions depending on which packed version in using.

Is it not possible for Ipopt.jl (for example) to come out with a new binary (for whatever reason) so that even if I use the same Julia version, I’m still referring to different binary dependencies?

That’s for the idea about renaming the .julia folder. I specifically did a fair amount on reading in updating Julia (this was my first time), and the consensus server to be that this sort of thing is not necessary. But my experience send to suggest otherwise

erans · April 7, 2021, 1:44am

I’m not sure unfortunately, I can run the test again tomorrow.

There are several dependencies that area failing though, not just that one. I also had a similar issue when trying to add Ipopt.

This was done from my home network, so no firewall or anything like that.

Oh, I also tried deleting the complied folder in my ~/.julia folder, but that didn’t seem to make any difference either.

odow · April 7, 2021, 2:36am

In general, the process to update Julia is to download new binary from julialang.org, and that’s it. You don’t need to mess with ~/.julia.

I would try again, but make sure you delete /tmp/Project.toml and /tmp/Manifest.toml first. Problems can arise if you instantiate a Julia 1.6 environment in a folder containing a Manifest from Julia 1.5.

erans · April 7, 2021, 9:52am

Interesting. This is infact exactly what I did following this thread:

Is the problem then that I not only copied but then also ran:

@v1.6 pkg> instantiate

If that’s the case, what is the correct way to transfer packages from the base environment of one version to the next?

kristoffer.carlsson · April 7, 2021, 9:53am

You can copy over only the Project file (leave the manifest) and then do instantiate. Some changes have unfortunately made manifests from e.g. 1.5 incompatible with 1.6.

erans · April 7, 2021, 9:55am

Got it. That make sense.

Would you recommend reinstalling Julia 1.6, or can I simply remove the Projetct.toml and Manifest.toml from the v1.6 environment?

kristoffer.carlsson · April 7, 2021, 9:55am

No need to reinstall Julia. Just remove the Manifest and do instantiate and it should hopefully work fine.

jdad · April 7, 2021, 11:09am

Would ]update do just the same thing as removing Manifest+Instantiate? (Plus added bonus of keeping non-registered added packages and pinned indications?)?

Henrique_Becker · April 7, 2021, 1:59pm

I do not think so. Update probably looks at the Manifest (to check if it needs to do any work or not) and if the Manifest formats are incompatible there a chance you will get a problem this moment. If you check the code of Pkg.update it calls prune_manifest and other methods that seem to read from an existing Manifest (if it exists).

jdad · April 7, 2021, 2:17pm

But in case of 1.5 → 1.6, there is no format incompatibilities AFAIK, only the fact that some jll that were previously independents are now embedded in Julia1.6 as stdlibs. So the issue is only to update those jll to the right ones?
Possibly I should have said ]resolve that update thé dependencies (while ]update goes to the latest ones) - in any case, ]update worked for me in my similar trouble cases.

kristoffer.carlsson · April 7, 2021, 3:25pm

Hm, it might be that 1.5 → 1.6 is ok but that 1.6 → 1.5 is the really bad case actually. I am not sure.

erans · April 10, 2021, 10:06pm

This seems to have been the issue.

I deleted my v1.6 Manifest.toml and ran resolve and then instantiate (leaving out the resolve led to Errors since I had missing dependencies in the (non existent) Manifest).

When I now repeat my original “Step 1” everything works.

Thanks for the help!

compleat · October 19, 2021, 11:23am

Hi I am teaching a class and my students are having the problem that attempting to add JuMP just hangs for Windows users [Mac users are fine] in the latest Julia 1.6.

Does anyone have any ideas?

Thanks

erans · October 19, 2021, 8:18pm

Could you maybe include the REPL output up to the point where it hangs?

My issue was not with hanging but with errors, and that was caused by mixing Julia 1.5 manifest with Julia 1.6.

If your students are installing a clean version of Julia 1.6 from the website and then doing ]add JuMP it should work just fine (I’ve done it several times in several windows machines)

odow · October 19, 2021, 8:48pm

Please prefer to open a new post if you have a related question, rather than posting on a issue that was previously solved.

Does this only happen with JuMP? Have they tried other packages?

compleat · October 20, 2021, 10:03am

Thanks. There was no output - it just hung without anything. I’m sorry if this is unrelated, but I wondered if two difficulties installing the same release of the same package might be related. Also, I know that it is impossible to troubleshoot without output, but I wondered whether others had experienced the same issue [apparently not - thanks very much for checking!]

erans · October 20, 2021, 4:26pm

If it didn’t even output the message saying that it’s updating the registry index, then I’d suspect there is something wrong with the Julia installation itself and second the question about what happens with other packages.