Status of new package manager?

Hello all. I haven’t heard anything about the new package manager lately, I was assuming this would debut in 0.7, but I don’t see anything about it in NEWS.md. As we all know the old package manager can frequently cause pain, so I was wondering if we were going to see anything new in 0.7.

https://github.com/JuliaLang/Pkg3.jl

Try it out! It’s awesome!

6 Likes

Cool. Any word on whether this will be in 0.7?

Since the package names are hashed it’s hard to know how to work with this before any documentation is written.

I have tried Pkg3 recently. The architecture is amazing, it is fast, convenient, and does the “right thing” in many situations. For example, it hooks into using and asks whether it should install packages that are not available. Updating and adding packages is much faster than Pkg2.

That said, there are still some minor issues and open design questions awaiting decision. Specifically, even though I was very impressed by the speed and convenience, I reverted to Pkg2 because of issues with local packages.

I am not sure I would prefer Pkg3 to hold up v0.7. I think that the latter is focusing on introducing all the breaking changes into the language.

It will be in 0.7/1.0. There’s a help command so if you follow the README instructions and do ] to get into Pkg3 REPL mode and then type help you’ll get some commands. It definitely needs some documentation, but we’re still working hard at finishing it…

5 Likes

Unfortunately, they are not unrelated so we don’t really have a choice but to couple 1.0 and Pkg3. A key part of the Pkg3 design is the ability for different dependencies to load different packages by the same top-level name. In other words your code can do import Foo and load one Foo package while one of your dependencies can do import Foo and load a different Foo package. This is absolutely necessary to support federated registries in a way that doesn’t break code in unpredictable ways and requires re-architecting the way code loading works in a breaking way. One of the implications is that packages will not be able to load undeclared dependencies, even if the dependency is already loaded. Of course, they shouldn’t be doing that in the first place, but I’m sure that many are doing so. I also think that 1.0 is a good time to make a clean break from one package manager to the new one, which has a rather different model. In fact, the issue you linked to and this one are pivotal: we may want to get rid of LOAD_PATH entirely, which we can’t do if it exists in 1.0.

2 Likes

Awesome. Thanks for all your hard work. I’m sure that spending so much time writing a package manager could not have been fun by any means, I’m sure we all really appreciate the result.

I also think it will be really good to have this as default in 0.7 so that everyone is able to get on the same page before the 1.0 release.

2 Likes

Currently packages and modules are not strongly coupled; using and import can work with whatever is in the load path. Are you saying that this will no longer be the case?

I can see the advantage, but it would be nice to keep the option of having various directories with packages that exist nowhere but on my hard disk (and of course the backups :smile:). Packages are lightweight enough and easy to create so that I don’t mind setting up a bit of extra wrapping for some modules.

I can’t say I haven’t occasionally regretted “signing up” for the job, but it’s not always so bad. Early design discussions with @wildart were really fun and lately seeing it actually working and having people like @kristoffer.carlsson and @fredrikekre helping carrying on with the work is really gratifying. I hope it lives up to all the expectations.

9 Likes

Currently packages and modules are not strongly coupled; using and import can work with whatever is in the load path. Are you saying that this will no longer be the case?

This doesn’t change the relationship between packages and modules at all. It would only change the way files are found that might be auto-loaded when import Foo is encountered and Foo isn’t already defined.

I can see the advantage, but it would be nice to keep the option of having various directories with packages that exist nowhere but on my hard disk (and of course the backups :smile:). Packages are lightweight enough and easy to create so that I don’t mind setting up a bit of extra wrapping for some modules.

See look in LOAD_PATH for packages by fredrikekre · Pull Request #53 · JuliaLang/Pkg.jl · GitHub for more discussion of this. It would still be possible to use local directories and files, but it would be done more explicitly rather than implicitly – i.e. have a path to your code in the [deps] stanza. I promise this will be worked out before feature freeze.

2 Likes

9 posts were split to a new topic: REPL modes don’t work in Juno

Yes.

Is there still a way to specify local dependencies like for example now with a REQUIRE file in the test-directory?

I’m curious about how we edit source code now that everything is stored in these hashed directories. Do we keep it somewhere else and Pkg3 copies it?

1 Like

I attempted to try it out yesterday, on a freshly built master, but it didn’t work (at least, the ] character to switch to the Pkg3 REPL mode didn’t do anything).
Is this only working for the v0.6.1 release currently?

Also, one of the instructions needs to be updated:

First, Pkg.clone the Pkg3 repo itself to install it:

Pkg.clone(“GitHub - JuliaLang/Pkg.jl: Pkg - Package manager for the Julia programming language”)

Really looking forward to being able to use this!

Did you do using Pkg3 first?

Yes, of course!