When a package is registered for the first time, it looks like there is a bug where the uuid in the Project.toml file is not respected, and a new uuid is placed in the JuliaRegistry/Uncurated.jl repo, which breaks any following packages that are in the process of getting registered, and means that a new tag is needed for the newly registered package as well, just to replace the uuid with the “new” uuid.
We do not support registering packages to the registry yet. The old METADATA is still the source of truth for package registration and all UUIDs are assigned determinisrically by name.
Why ignore the uuid, if there’s one already in the Project.toml file?
How can one generate the same uuid that the METADATA → Uncurated.jl process creates, to avoid this problem of having to tag another release, just to get the uuids to match?
There’s a lot of machinery for the transition that relies on those predictable UUIDs. There is no need to put a project file in your registered packages yet. You literally do not need to do anything different.
I wasn’t asking about registered packages. The issue is with packages that are still in the process of getting registered. Pkg3 is having a lot of problems dealing with them, even when all of their dependencies have already been registered. (There’s a catch-22 situation where says it can’t find the package in the current directory, but if you try to “add” it, it says it can’t add the same name/uuid to the current project).
I think Stefan is saying that you should stick with the old package manager for registering packages currently
Correct.
I thought part of the point of having the alpha release was that package maintainers could start transitioning to things like Pkg3, and make sure things were passing unit tests without deprecations on v0.7.
How is one supposed to use the old package manager with v0.7 then, in the meantime?
Before Pkg3 dropped with v0.7.0-DEV.5183, I had all of the packages in JuliaString.org passing on both Linux & Mac, v0.6.2 & master on Travis, but they’ve been broken on master even since (even using the default Travis build script).
Hmm, I just ran some tests for ReduceLinAlg on 0.7 and also ran into some issues. Specifically, it depends on another package Reduce which also depends on Compat and even though I install it, the package can’t be compiled since the Compat says it’s not installed. So Scott is right, there are some weird things going on.
Reproducible examples filed as issues are the best way to get things fixed.
Can’t, at least not in JuliaLang.
This shows it happening on Travis, using the default build script: Travis CI - Test and Deploy with Confidence
I think I’ve realized what’s causing the other problem that has been coming up.
When a package is registered or tagged, it picks up the requirements not from an existing Project.toml file in the top-level directory of the package, but rather from the old REQUIRE file.
For my packages, I only had the minimum necessary to assure all requirements were met (i.e. REQUIRE for StrLiterals had StrBase, but not all of the things that StrBase directly or indirectly depends on, and it seems that the new package manager really wants all of them (at least, any that might be present in a using or import statement) to be in the Deps.toml file in the registry.
It seems like, if it does want everything in the Deps.toml file (like it does in a Manifest.toml file), then if it’s going to pick up from the preexisting REQUIRE and ignore the Project.toml, it needs to recursively go find all of the other depencies that are indirectly required.
There are a lot of chicken and egg problems involved in switching a live package ecosystem to a new package manager. Normal software upgrades are a child’s play compared to this. Please be patient and note the following:
-
You do not need to do anything for existing packages. We will make PRs to registered packages to add project files to them with the appropriate contents.
-
During the transition, we do not support you adding your own project file and then trying to register a package. If you try to do this it will break.
-
The old METADATA.jl repo is still the source of truth for everything about the registered package ecosystem. Do not try to register anything at the new
Uncuratedpackage registry. That will eventually be the source of truth, but until it is, it is just a replicated copy of METADATA.jl in a different format that the new package manager understands.
4 Likes
Everything you do import or using on need to be in the REQUIRE file. The new package manager does not need all deps recursively in the project file.
This was what had not been communicated.
Is there anything where that has been stated, for package authors / maintainers trying to move to v0.7.0-alpha?
OK - I’d already taken the time to put in useful information there, such as the authors and keywords fields.
OK, I’d assumed from the Pkg3 documentation, that I just had to have those deps listed in the Project.toml file, which I did.
Sorry that this hadn’t been communicated clearly. Hopefully it is clearer now.
3 Likes
Yes, that is correct. All the direct dependencies have to be in the Project file if you have a project, otherwise in the REQUIRE file. So not recursive.
1 Like
Even with removing the Project.toml file, and making sure that the REQUIRE lists everything that is imported or used, and using the default test script, I get the following:
15.01s$ julia --color=yes -e "Pkg.build(\"${JL_PKG}\")"
WARNING: Base.Pkg is deprecated, run `using Pkg` instead
in module Main
Building PCRE2 → `~/.julia/packages/PCRE2/zl3H/deps/build.log`
40.84s$ julia --check-bounds=yes --color=yes -e "Pkg.test(\"${JL_PKG}\", coverage=true)"
WARNING: Base.Pkg is deprecated, run `using Pkg` instead
in module Main
WARNING: Base.Pkg is deprecated, run `using Pkg` instead
in module Main
Testing StrRegex
Resolving package versions...
ERROR: LoadError: ArgumentError: Module ModuleInterfaceTools not found in current path.
Run `Pkg.add("ModuleInterfaceTools")` to install the ModuleInterfaceTools package.
Stacktrace:
[1] require(::Module, ::Symbol) at ./loading.jl:868
[2] include at ./boot.jl:314 [inlined]
[3] include_relative(::Module, ::String) at ./loading.jl:1071
[4] include(::Module, ::String) at ./sysimg.jl:29
[5] top-level scope
[6] eval at ./boot.jl:316 [inlined]
[7] eval(::Expr) at ./client.jl:394
[8] macro expansion at ./none:3 [inlined]
[9] top-level scope at ./<missing>:0
in expression starting at /home/travis/.julia/packages/ChrBase/5Q6a/src/ChrBase.jl:11
ERROR: LoadError: LoadError: Failed to precompile ChrBase to /home/travis/.julia/compiled/v0.7/ChrBase/hTjE.ji.
It seems the problem is that the registered ChrBase (v1.1) does not declare a dependency on ModuleInterfaceTools even though it is using it.