Reduce package registration waiting period

Currently there’s a waiting period of 3 days before a new package is automatically merged into Julia general registry. I feel like the 3 day length was arbitrarily chosen and that it isn’t really required to be that long. Especially for a relatively young ecosystem, as it imperils the package growth.

Also, almost all other package registries have a waiting period of 0 seconds (PyPI and npm for example).

What other arbitrary time would you suggest? Is it really that much to ask to wait 3 days for a package that hopefully will be used for years? When are you ever in a hurry to release a new package?

17 Likes

I’m not sure I see who this would benefit. The person who is publishing the package already has access to it. Anyone else who can’t wait the three-day registration period can easily add the package by URL.

6 Likes

Waiting seems pretty easy to do, no?

24h seems to short. Given that the idea for the waiting period is for people to chime in I don’t think anything shorter than 3 days is long enough. Can’t expect people (maintainers) to be online 24/7.

13 Likes

Using the same logic, you can arrive at the conclusion that it is ideal for packages not to be published at all. The person who is publishing the package already has access to it and anyone else who can’t wait the three-day registration period can easily add the package by URL. Am I not right? :smile:

I think it benefits both the author and the potential users. First, author doesn’t have to use ]activate . or similar commands repeatedly. And second, other people can access it more easily as well. It contributes to the general ease of use.

1 Like

Just to clarify, this is for new packages only – new versions of existing packages are merged immediately.

10 Likes

Ideally, packages that can be merged automatically should require no maintainer input at all.

I understood that the waiting period was implemented to give authors a chance to block the process, in case they think something is wrong. There is nothing related to maintainers.

Remember, npm package registry has more than a million packages. They have no issues with automatic merging into the registry.

I don’t think npm is necessarily a good model to follow though. leftpad etc.

9 Likes

My understanding is that the main purpose of the wait is to discuss the package name, or other things that might be improved; it definitely involves the maintainers and the (interested) community.

(I’m not advocating for a 3-day wait period, just pointing out what I think might be a misunderstanding).

I don’t use npm but, from reading reddit and hacker news, I get the impression that the npm registry policies have caused several significant issues…

5 Likes

No. Registration additionally handles name collisions and, although it doesn’t totally keep malicious code being distributed, allows for some safeguard against obvious attacks.

Conflicts is the word I was looking for. You can’t have two registered packages with the same name.

But if someone, say here on Discourse, post some cool code that I’d like to try out that has using SomeCoolPackage at the top, I’d like to be able to ]add SomeCoolPackage and run their example without necessarily having to worry about who wrote SomeCoolPackage. I know that’s not totally safe, but it’s at least better than nothing.

It’s really not a big inconvenience to wait for 3 days to initially register a new package.

The advantage of waiting for 3 days is that it gives the author time to think things over, and makes it so that new registrations are not spontaneous decisions.

Subsequent new releases usually get merged immediately, so that’s not a problem.

12 Likes

It doesn’t seem long for complex packages which require substantial amount of time to be built. However, it does seem long for packages that can be built in a day or two, or even in just a few hours. Effective build period in such cases is 4 or 5 days, instead of just 1 or 2.

Not all packages are registered as stable versions and most are registered as 0.1.0. Most are also just one file (look at https://github.com/juliaregistries/General/pulls). There isn’t much to think over with that.

The actual data doesn’t support such conclusions made in this thread. Very few, if any packages get merge-blocked manually and not automatically.

1 Like

I’ll put my name down as well on the side of the status quo.

If you think this is onerous, it’s actually pretty easy to set up your own registry, use that and even encourage others to use it in order to get out from under the oppressive boot of the 3-days in purgatory.

You could probably even make a registry that automatically scans the general registry for packages in purgatory and add them to your registry automatically.

9 Likes

The point I was making isn’t that a waiting period is going to solve these problems, it’s that the main purpose of the package registration system isn’t to make packages easier to add or use, it’s to provide users a “this is the official version of the package you are looking for”. That doesn’t really benefit from having a shorter waiting period since three days is pretty short compared to the useful life of a package. There is at least some benefit to an arbitrary, but not too short, waiting period because the most glaring attacks can be caught and it prevents users from too frivolously registering packages. Sorta.

I am opposed to shortening the waiting period for new packages.

There are plenty of examples in which members of the community have provided feedback on the name, and then the author has chosen to change the name.

Here is a single randomly chosen example: #15489

The author originally suggested the name GRPF.jl. The community provided feedback. One community member suggested the name RootsAndPoles.jl. The author and the community engaged in a discussion. The author decided that they liked the name RootsAndPoles.jl. The pull request to register the name GRPF.jl (#15489) was closed. The package was registered with the name RootsAndPoles.jl (#15662).

If you go through the history of pull requests on the General registry, you will find other examples.


Keep in mind that the three-day waiting period only applies to new packages. New versions of existing packages are merged very quickly (within 20-60 minutes).

21 Likes

If anything, I think a current exploration of what’s in the “new” queue would argue for increasing the review time. I count a dozen new projects in the PR list that are 1) simple one-function packages, 2) without documentation, READMEs, or tests, 3) less than 100 or so lines of code, and/or 4) things that probably should be PRs to existing packages. (I realize the last one is subjective.)

I don’t know that it’s reasonable to require the maintainers to review these things for general registry merge within 72 hours, especially when there are other avenues for publication.

28 Likes

I don’t know much about other package registries, but this really doesn’t sound like a convincing argument, unless it’s demonstrated that the consequences of this policy were beneficial.

A counter argument to a zero-seconds waiting period: this wouldn’t give a lot of times to prevent an “attacker” from stealing a bunch of package names from the shared namespace. Of course there could be measures against that, but a three day waiting period is a natural such measure and has other advantages as pointed out by others.

2 Likes

I am not sure what you mean by this. Waiting three days is insignificant compared to the lifecycle of a successful package. If anything, the waiting period should be increased, to eg a week — not everyone is online 24/7.

Also, you may not be aware that Julia’s package manager has facilities for third-party registries. People who are effectively constrained by the 3-day waiting period for their own set of interdependent packages should probably just start their own local registry.

12 Likes

I’m one of “those” people: I was setting up tools for my lab, needed to have packages depend on each other, was publishing some of the basic ones, requested to register yet another package in my toolkit, and within the 3 day waiting time I managed to get useful feedback about some technical problems I was having (that package included a dependency on Matlab), was made aware of LocalRegistry.jl, retracted my registration (I think after like 40 hours), and now I and my lab use LocalRegistry all the time. It’s fantastic.

6 Likes