It takes several minutes to add a single package

Indeed, that is a problem. More reason then to be transparent to the user, and have he/she have complete control over the updates. If the user tries ups something and it does not get the latest version, we should tell him that, and that is it (the package manager does that, but only if the user wants to install a specific version). - or at least give the user that option, with a simple flag, like add -q, up -q.

I know when I’m using my linux distribution that many packages are being hold back because I do not have the latest distro. But I have the choice to when/how update things, without that messing up with my daily use of the system.

(and, as pointed by @Lilith, we have a situation where even adding a dummy package may trigger a lot of work, without having anything to do with any compat bounds).

I have a friend that had that kind of issue and 1.7 improved things a lot. It does not seem to be the same issue I have in that cluster.

Yet another candidate solution:

  1. Make ]add more conservative about updating unnecessary packages by default
  2. Add ]activate --reproducible name which prevents name from stacking with global (i.e. has the impact of removing @v#.# from Base.LOAD_PATH in only in name)
  3. Stop discouraging global environment usage

Where does it fall short of solving everyone’s problems? Where does it fall short of the status quo?

This was discussed in messages 44, 47, and some others.

It’s not working and falling back to getting the registry with git, i.e. exactly what it was doing before. Do you have some firewall or similar interfering? (I’m not an expert on SSL and certs stuff, hopefully someone else can help out with understanding that error.)

2 Likes

Not that I am aware of or have any control over. I will try to reach to the maintainers to see. If anyone has any hint on how to get more information, that would be nice, I would be able to provide them more data to track the issue.

One curiosity: is the new registry system ported to 1.6?

No. It’s a new feature, namely, the ability for Pkg to use a registry tarball without needing to extract the tarball.

As a general rule, new features are not backported.

2 Likes

Ooh I really like that.

@lmiq try setting a different package server

JULIA_PKG_SERVER=us-west.pkg.julialang.org

or some other one.

it is the same:

(@v1.7) pkg> registry add General
┌ Warning: could not download https://us-west.pkg.julialang.org/registries
│   exception = Cert verify failed: BADCERT_EXPIRED while requesting https://us-west.pkg.julialang.org/registries
└ @ Pkg.Registry /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.7/Pkg/src/Registry/Registry.jl:82
     Cloning registry from "https://github.com/JuliaRegistries/General.git"

Is the system time on your cluster badly miscalibrated?

Perhaps it thinks now is past the expiration date of the cert?

It is one hour in advance that it should be (now it is 17:27 here, and date reports 18:27). Can that be a problem? (reported already to the admin)

well, it seems like 1 hr shouldn’t be enough. When I connect to https://us-west.pkg.julialang.org I see

Expires on: January 17, 2022

Is it possible you misread the output of date and it’s actually off by a year or something?

nope:
% date
Tue Nov 2 18:26:26 BRST 2021

I’m not sure exactly, but I wonder if there isn’t a key exchange that occurs which is time-sensitive (like a symmetric key is negotiated and the far server says it’s good for 10 minutes) which then on the local end appears to be expired immediately. Get the time fixed, and see if it works! it’s definitely suspicious that the time is off!

Downloading github repositories seem to be particularly problematic. Any idea?

The tarball of julia downloaded now at 7Mb/s, but one repository downloaded at 280Kb/s (the same goes for the tarball of the release of that same repository).

It’s kinda unheard of to have a high performance cluster that has 7Mbps connection to the internet. So maybe talk with the administrators and see if they’ve got a serious networking issue. Like maybe there’s a network broadcast storm they don’t even know about? That’s definitely odd.

I wish. That is actually good… We don’t get much better speeds on anything involving the university link.

1 Like

Hunh… I’ve got a gigabit to my house… so I guess YMMV :stuck_out_tongue_closed_eyes:

2 Likes

This is where it would help if Pkg always told you if a more recent version was available but it wasn’t installing because reasons.

I have to say that I find it frustrating that Julia defaults to updating the registry, and potentially updates all of my packages (if new versions are available, which is often the case) whenever I want to install a single new package. I really only want it to update if I specifically run pkg> update — otherwise, Julia should assume I’m happy with the versions I currently have.

A workaround that does what I want is to put

Pkg.UPDATED_REGISTRY_THIS_SESSION[] = true

in your ~/.julia/config/startup.jl, but I wish this was the default.

13 Likes

One thing that might improve the user experience is to expand the documentation on environments, focusing on how to use them effectively, and not on how they are implemented. Currently, neither Julia’s nor Pkg’s manuals include the terms local environment or temporary environment. activate has only a short reference section in Pkg’s documentation.

3 Likes

I disagree. I think the best solution is making manual environment management less necessary rather than giving more instructions on how to do it.