Some more observations on working with Pkg(3)

[kristoffer.carlsson]

@ExpandingMan

By the way, one more question, if one does add SomePackage , is the dependency added fixed at whatever was the most recent version when the command was called?

Whenever you add a package, a resolution is run that determines the latest possible version of that package, which will be installed. Same when you upgrade. When you have a Manifest and you do instantiate you install the exact versions of the packages recorded in the manifest without any resolution.

[ExpandingMan]

@kristoffer.carlsson

Should be easy to revive though.

I made a comment to that effect in the PR. I think this would be extremely useful for container setup scripts such as dockerfiles.

@kristoffer.carlsson

When you have a Manifest and you do instantiate you install the exact versions of the packages recorded in the manifest without any resolution.

So essentially what I think you are saying is that you have to explicitly tell it to upgrade a package every time you want one updated? In other words, if I create a package and doe add, and it fetches me SomePackage 0.1.0, then my package will always use SomePackage 0.1.0 until I do up SomePackage from its environment?

[StefanKarpinski]

pkg> status --tree mode is a good idea; issue filed:

https://github.com/JuliaLang/Pkg.jl/issues/318

[StefanKarpinski]

@ExpandingMan

On that note, one thing that seems to be missing is some sort of command line entry point to the package manager.

Currently the issue is that Julia startup + Pkg load time is a bit too heavy to make this very pleasant to use, so you’re better off having keeping a package manager REPL session going. But in the future it’s certainly something we could have.

[StefanKarpinski]

@ExpandingMan

So essentially what I think you are saying is that you have to explicitly tell it to upgrade a package every time you want one updated?

Yes, Pkg3 is much more conservative about upgrading packages. It never upgrades direct dependencies unless you explicitly ask for them to be upgraded; it will upgrade indirect dependencies as necessary, but tries to avoid it if possible.

[ExpandingMan]

@StefanKarpinski

Currently the issue is that Julia startup + Pkg load time is a bit too heavy to make this very pleasant to use

I think that for the primary use case (set up scripts) one doesn’t have much choice and/or it doesn’t matter. As things are right now it seems that docker files might have to be a little ugly

julia -e "import Pkg; Pkg.add(\"PackageAlpha\"); Pkg.add(\"PkgBeta\"); Pkg.checkout(\"PackageGamma\")"

@StefanKarpinski

Yes, Pkg3 is much more conservative about upgrading packages

I suppose that is the canonical approach, but I have to say this makes me a little bit nervous especially in Julia’s case. Since it is a young language, updates are much more important than they’d be for e.g. Python and now Pkg provides people with tools that makes it very easy for them to get stuck on legacy code. If it were me I’d probably make the default behavior to get the most recent packages unless you explicitly cap it, but I recognize I probably have the minority opinion on this.

[jonathanBieler]

I haven’t read everything but I’ve one question and one remark:

When trying to update packages to v0.7 having to open log files for failing build is quite annoying, is there an option to show the log in the REPL ?

Some colors (light yellow and green) can be very hard to read, choosing colors with similar luminance would be good.

[StefanKarpinski]

The Pkg2 behavior of upgrading things that one didn’t ask it to upgrade is incredibly annoying and ends up forcing a lot of unnecessary pinning. (JuliaPro ships with everything pinned because this behavior is so problematic.) If I’ve asked the package manager to upgrade X then it should upgrade X, not some unrelated package that I didn’t ask it to upgrade. I don’t think the fact that people have to do pkg> up to upgrade everything will really prevent them from upgrading their packages.

[StefanKarpinski]

@ExpandingMan

julia -e "import Pkg; Pkg.add(\"PackageAlpha\"); Pkg.add(\"PkgBeta\"); Pkg.checkout(\"PackageGamma\")"

You can do this instead:

julia -e 'using Pkg; pkg"add PackageAlpha PackageBeta PackageGamma"'

Edit: changed import to using as pointed out below.

[ExpandingMan]

@jonathanBieler

Some colors (light yellow and green) can be very hard to read, choosing colors with similar luminance would be good.

It depends on your background, I’m assuming the vast majority of users probably have dark backgrounds.

@StefanKarpinski

The Pkg2 behavior of upgrading things that one didn’t ask it to upgrade is incredibly annoying and ends up forcing a lot of unnecessary pinning.

Yeah, I know it was going to have to be this way, this was more of a general expression of angst than a serious suggestion.

@StefanKarpinski

I don’t think the fact that people have to do pkg> up to upgrade everything will really prevent them from upgrading their packages.

Ever the optimistic idealist!

@StefanKarpinski

You can do this instead:

Great, thanks. It might be good to feature this prominently in documentation, since this will be a very common thing for users to have to do because of set up scripts.

[goedman]

Earlier I asked about the extra layer in .julia/packages/PkgName/xxxx (item 10) and Kristoffer answered. I assume it is directly related to the phrase “in different locations on your system” and later on “garbage collected” in the Introduction section of the Pkg docs?

The Introduction also mentions “stackable”. Is it correct that when one executes add PkgName it is really (implicitly) a fresh copy of project PkgName that is added? At least if a Project.toml is present. In that case a generic up updates all packages, including (and only) the local copy of the added project? I.e. up is recursive but local.

The 3rd section in the Introduction is what Scott was asking about earlier I guess.

[ScottPJones]

@StefanKarpinski

julia -e ‘import Pkg; pkg"add PackageAlpha PackageBeta PackageGamma"’

Minor point, that doesn’t work as written, you need to do julia -e 'using Pkg; pkg"..."' so that the @pkg_str macro can be used.
(I do love being able to do that, it’s a very nice addition!)

[kristoffer.carlsson]

It is perhaps better to give a vector of strings to Pkg.add instead. Will give the same result but doesn’t use the repl interface which is arguably made for interactive use.

[ScottPJones]

I wasn’t even aware that it could take more than just a single string now.
Thanks for letting us know!

[ExpandingMan]

Pkg3 seems broken for me on the 0.7 alpha.

For every package I get

(v0.7) pkg> add JuMP
ERROR: The following package names could not be resolved:
 * JuMP (not found in project, manifest or registry)
Please specify by known `name=uuid`.

even though the registry seems to be full. Any ideas what’s going on?

Edit: I seem to have fixed it by deleting the registries directory. So if you encounter this problem you might try that.

[ExpandingMan]

Here’s another question:

DataFrames is itself fully updated for 0.7, but it pulls an old version of StatsBase. What’s the best way of getting it to use StatsBase master? Right now the only way I see of doing that would be dev DataFrames, then going into DataFrames and updating. Is that indeed the recommended procedure, or is there some way to tell the tagged version of DataFrames to use the master of StatsBase? (i.e. without doing it manually in git)

[StefanKarpinski]

pkg> add StatsBase#master

Edit: actually, that adds StatsBase to your project file which you don’t really want since its not a direct dependency. Another way to do this is pkg> dev StatsBase but that checks out a git clone at ~/.julia/dev/StatsBase. I can’t think of a way to use the master version without either adding it to the project file or checking out a dev copy. Perhaps we should have a way of doing that.

[ExpandingMan]

Having to do dev DataFrames to update the StatsBase doesn’t seem totally unreasonable to me, but I can imagine a situation in which I’m working on StatsBase, and I want to check to make sure I haven’t screwed up other packages. Right now I’d have to dev on each of those packages.

Another question: is there away to generate the toml’s for the legacy 0.6 packages? That would be really useful especially since they contain the uuid’s.

[miguelraz]

I had this happen and I solved it by removing old installs of v0.7 in my ~/.julia/ folder.