Incompatible changes in Pkg?

General Usage
1

I am getting some new errors. With 1.5.3.

Current folder: C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\tests\FinEtools.jl                                                                                                                
 Activating environment at `C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\tests\FinEtools.jl\Project.toml`                                                                                      
 Installing known registries into `C:\Users\PetrKrysl\.julia`                                                                                                                                           
      Added registry `General` to `C:\Users\PetrKrysl\.julia\registries\General`                                                                                                                        
ERROR: LoadError: TypeError: in typeassert, expected VersionNumber, got a value of type Pkg.Types.VersionSpec                                                                                           
Stacktrace:                                                                                                                                                                                             
 [1] load_urls(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Operations.jl:510                                   
 [2] #download_source#54 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Operations.jl:686 [inlined]                                                                      
 [3] download_source at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Operations.jl:685 [inlined]                                                                          
 [4] instantiate(::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Pkg.BinaryPlatforms.Windows, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:875                                                                                                    
 [5] instantiate(::Pkg.Types.Context) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:795                                                                          
 [6] #instantiate#169 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:791 [inlined]                                                                                
 [7] instantiate() at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:791                                                                                             
 [8] top-level scope at C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\testall.jl:29                                                                                                             
 [9] include(::String) at .\client.jl:457                                                                                                                                                               
 [10] top-level scope at REPL[1]:1                                                                                                                                                                      
in expression starting at C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\testall.jl:24

No clue what is going on since this used to work with 1.5.2. I don’t know what to look for an error. Any ideas?

Edit: it looks like it might have something to do with the Manifest.toml update which I carried out for 1.6. It would appear that at least some packages are not backward-compatible from 1.6 to 1.5.x.

Edit 2: It was definitely that. 1.6 will make the Manifest.toml incompatible with 1.5.

2

A Manifest.toml file generated by one version of Julia is not guaranteed to be compatible with any other versions of Julia.

3 Likes
3

Yes, I just found out the hard(er) way. :wink:

4

What happens is that some packages moved from being normal packages to being stdlibs in 1.6. That wasn’t really expected, so Pkg gets confused now when it sees a manifest with an stdlib that gets run on a version of Julia where that package is no longer an stdlib.

3 Likes
5

Will this change break applications I made for reproducibility or does it only affect packages?

Is the fix from my end to erase the manifest.toml and start over? That’s ok for packages since one doesn’t put the manifest on the repo anyhow. It is less ok for reproducibility since those applications are supposed to be archival.

6

(Strict) reproducibility assumes you are running with the same Julia version.

3 Likes
7

Of course, Duh! So if Project.toml has something like

[compat]
julia = "1.5"

will that fix it up?

8

That will allow any version in [1.5, 2), which isn’t what you want here. Instead, you probably want "~1.5" to limit to versions 1.5.x, per 6. Compatibility · Pkg.jl

4 Likes
9

Hello, I hope it is not a problem if I add my question to this (relatively) old post.
I got the same error in a continuous integration context: I edited the package using Julia 1.6, and the changes also implied a change of the manifest. This then caused CI to fail because ci.yml was still checking on Julia 1.5.
My question is: is the new version of my package absolutely incompatible with julia versions < 1.6 ?
Or can I try to track down which packages/changes caused this and attempt to fix it?

10

If you check the manifest into the repository and want to run CI against v1.5 you have to always generate/update it with Julia v1.5 to ensure backward compatibility. Alternatively, remove the Manifest from the repository and you’ll live much happier :slightly_smiling_face:

(To be clear, I recommend deleting the manifest only for packages, not “applications” which can be for example an analysis pipeline)

1 Like
11

Thank you, I will just delete it and “gitignore it” in the future then :slight_smile: !

1 Like
12

However, be aware that you might be giving up on reproducibility: According to the Manual
The manifest file is an absolute record of the state of the packages in the environment. It includes exact information about (direct and indirect) dependencies of the project. Given a Project.toml + Manifest.toml pair, it is possible to instantiate the exact same package environment, which is very useful for reproducibility.

13

Which you usually don’t care about when testing a package: you want to test on as a wide range of versions as possible, not just a single fixed one

14

Indeed. But when someone uses a package, they might care about reproducibility a lot.

15

That’s why I wrote

16

But packages are typically constituent parts of applications, and they well may be used as applications on their own.

17

You use appropriate compat bounds to make sure things don’t break.

3 Likes
18

So, if I understood correctly, removing the Manifest file and adding it to gitignore for next time on the package allows it to be teste by github actions on different Julia versions (and therefore different packages) ?

On the other hand, on the project that has this package in its dependencies, the manifest is still there and still contains speciic versions of the dependencies of the package, so If i forward this projet to someone reproducibility is still ensured ?