Revoking packages - or any registration obligations

Community
1

Continuing the discussion from [ANN] Kaimon.jl — opening the gate between AI agents and Julia:

I tried to look this up but couldn’t find what you refer to (only like OSI requirement, and naming guidelines). It could be you mean like an ethical obligation.

Context, a Github repo was regrettably deleted (of an awesome registered package or two), or hidden. I checked and I could still install, and use (using ...), and I suppose also all older versions can be installed (if I had knowledge of old version numbers, can’t see them, I think they also must be there in the registry; without much context):

How do I remove a package or version from the registry?

You can’t. Package registrations are permanent. A version can not be overwritten in the registry, and code cannot be deleted.

[To prevent the “leftpad” situation. Exception is “yanking” versions for security. I suppose if an author felt he violated rules he might drop his repo, not saying that happened here.]

I DID ask AI what the obligations are, anyone can do it (i.e. I simply asked Google, looked at its AI answer), and got a seemingly very good answer, and I also scanned all the docs I could find. I hesitate to post the answer (though I actually did, then deleted it, if curious you can see version one of this answer) since AI [answers] are a hot topic right now…, it was at least not showing any obligations about removing a package, nor did I locate myself, so I would want this clarified.

There certainly are no obligations of developing a package further; or can be for many reasons. In extreme cases like this (or say in case of death), is there after some time a way to claim the name and package for further development? Or such procedure not yet decided or has been discussed.

2

Yes, I mean entirely an ethical obligation.

At a fundamental level, the consensus of registry maintainers is that the General registry is a “community resource”. There is no list of obligations that this entails, just common sense and the general community code of conduct, which boils down to “don’t be an asshole”.

At the most immediate level, the General registry being a community resource means that new registrations are subject to community review (anyone can flag concerns about a package during the mandatory three-day waiting period). As a consequence of the naming guidelines, “descriptive” package names are the norm. This makes good package names scarce: there can only be one authoritative package for a particular domain (by name), and that creates the continuous concerns about appropriately named packages, as well as expectations for the continuing maintenance of those packages. A package like DifferentialEquations advertises an extremely broad scope and has the corresponding expectations (which SciML can meet) while the equally descriptive but much more narrow GroundHeatExchanger (random recently submitted package) would receive much less scrutiny.

Post-registration, the responsibilities of a maintainer a more nebulous, and there really isn’t much (if any) of an enforcement mechanism. So, whatever common-sense ethical obligations you can think of. Basically: maintainers should maintain their registered packages in good faith and to the best of their abilities. What it means to me is that I expect maintainers to accept reasonable contributions that are within the scope of the package (the more general the package name, the more important), and keep up with basic maintenance to make sure the package keeps functioning. If a maintainer can’t or doesn’t want to keep working on a package, they have an obligation to pass the package to a new maintainer, if and when someone else wants to step in.

The AI in the “version 1” of your post was spot on when it said “commitment to maintain”, in addition to the expectations and requirements laid out in the General registry README.

When maintainers just disappear and package thus become abandoned, registry maintainers can and do step in to assign the package to a new maintainer, by switching the package URL to a fork (which we’ll typically insist will have to be owned by an organization, not an individual account). This is a last resort, and will never happen over the objection of a maintainer. If at all possible, we try to have the existing maintainer facilitate the handover to a new maintainer.

Deleting a registered package is not possible, and deliberately deleting the underlying repo feels to me like a violation of ethical responsibilities (but maybe that’s just me). At a technical level, the registry is robust against this sort of deletion: the package servers cache all versions of all registered packages. Indeed, this robustness and reproducibility guarantee is what distinguishes the General registry from other registries. What should happen in such a case, in the long term, would have to be figured out by consensus of the registry maintainers. I could see a fork being allowed to register. Under normal circumstances, the registration of forks receives a lot of scrutiny, and generally will not happen without the consent of the forked package maintainer.

As a most extreme hypothetical ethical violation, if a maintainer started pushing malware through their package, I could see the registry maintainers stepping in to lock them out of the registered package. So far, this has never happened. In general, any issue with registered packages will be decided on a case-by-case basis, and with a light touch: we want package maintainers to maintain their packages as they see fit.

3

It’s not just you. I’m not even close to being in the know, but I will say that removing a repo from the public leans toward the worse ways of walking away from a community that mostly had no involvement in the circumstances. I don’t believe that FOSS developers are inherently obligated to provide their work forever, but taking action to opt out of the numerous ways to pass the torch contradicts the prior participation in a registry where persistence is the point. Granted, I’m highlighting one mistake among many, some of which I probably lack the information to recognize, but a comprehensive catalog of the world’s failings isn’t necessary to recognize or address a particular one.

I wonder if General registration is encouraged too much. I get that Pkg doesn’t resolve unregistered dependencies*, but making new registries that build on General sounds like a decent option for sets of packages that don’t have the same goals as General. But I haven’t even bothered to use a local registry in practice so I don’t really know how reasonable that is. I just personally wouldn’t mind adding a couple more registries to install packages, though an unfettered explosion into adding dozens of registries would be revolting (imagine if add Plots becomes easily ambiguous).

*Some package managers download package metadata on demand instead of a monolithic registry/index at once like Julia, and that sounds like it could also apply to unregistered Julia packages. (I dislike typing a lot when adding packages, but is a repo link and a Project.toml enough metadata?) However, metadata on demand brings its own issues, and while I’ve heard of super-fast approaches e.g. Astral’s uv for PyPI (which does seem to rely on a centralized metadata directory per index for a fast path, so not exactly unregistered), I don’t know if that could be appropriate for Pkg.

4

Just to be absolutely clear about this aspect: I do not think that any maintainer owes anyone their time. Unless you’re paying someone, you don’t get to tell them what to work on. The single most important moral responsibility I would put on a maintainer is to allow others to step in when they no longer have the bandwidth to manage the project. I would generally encourage people to give commit access to frequent and trusted contributors. Also, popular packages are best moved to a GitHub organization, since that allows multiple owners and makes it trivial to switch maintainers

I wonder if General registration is encouraged too much.

I think it is. There is of course the major factor that packages that are registered in General can only depend on other packages that are registered in General. The reason for this is the strong reproducibility guarantee that the General registry makes: if a package depended on a non-registered package, and the repo of that non-registered package disappeared, the dependent package would no longer be installable. But that also means that if your package is not in General, its reach is inherently limited.

Still, most packages are not used as dependencies, and any such package that is still at an early stage of development does not need to be registered. Honestly, with my current perspective, I would probably not register many of my own packages that aren’t 1.0 yet. That’s actually been something that’s been floated as a requirement. At some level, “1.0”-ready (with all that entails according to semantic versioning, most importantly a complete API documentation) is a good guideline. We’re not going to make that a rule, partly because it’s a much higher bar and we don’t have the tooling to enforce that higher level of quality, but also because at least I would always register at a v0.x level as a “release candidate” for a soon-to-follow 1.0 release, once the API has proven itself. But it’s a good rule of thumb: registered packages should be polished (and well-maintained) enough that they can serve as reliable dependencies to the larger ecosystem. In that context, pre-1.0 vs post-1.0 version numbers mainly signal “how strongly is the development focused on maintaining backwards-compatibility”. The registry already enforces this by requiring release notes detailing “breaking changes” for v0.x releases, which is a policy that deviates from semantic versioning. I’ve made my peace with that policy based on the idea that if you feel you can’t document your API and track breaking vs. non-breaking changes (since semantic versioning really starts at v1.0 and there is no specified API that could “break” in a pre-1.0 release), then you should hold back on registration.

5

Absolutely, 100%. I really wish it wasn’t. I really wish there was a registry that had equally strong committent to reproducibility, but also substantially higher entry barriers, and multiple of them. It would make wonders for the security of the language and its adoption. I wish there was a reproducible, stewards-backed registry, that did enforce rules instead of shying away from them. Anyways this is another discussion alltogether, but i hope my post doesn’t land too much off topic.

6

I get the v1.x.x ideal but I don’t think that’s realistic. Many widely used packages like Distributions.jl, PDMats.jl, StatsBase.jl, Clustering.jl, Flux.jl, Makie.jl aren’t there yet. Some move incredibly slowly despite being pre v1, because they don’t want to break anything. I mean Distributions.jl and StatsBase.jl have been around for ages and underpin large parts of the ecosystem and are still v0.

And then there’s the SciML ecosystem which has packages on v8 or something and accidentally breaks in minor releases all the time, which is reasonable given their scope and speed of development.

That’s not to mention that a large number of heavily depended on packages lack examples, and their APIs are not very well documented, if at all. Some defend their lack of proper docs by claiming a given package is aimed at developers of other packages, which is bonkers if you ask me, and it all but guarantees that other people will be frustrated when trying to use them, and must resort to reading esoteric source code.

7

If my wildest hopes about Pkg resolving unregistered dependencies after downloading the minimal uncentralized metadata on demand is actually feasible, then the reach could get very far outside the mainstream, albeit more vulnerable to dead links. But anything that becomes mainstream should find its way into one central registry, if only to make packages easier to find and install by name. Emphasis on one, because worst case scenario we’re all depending on each other’s personal registries, which isn’t better than somehow making unregistered resolution work.

It sounds nice to have a GoldStandard distinction from General, but part of the problem is that reproducibility conflicts with the implied enforcement of quality. Packages don’t often start out really good, and they often decline with no guarantee of resurgence. For a non-existent example, a project with Webscraper.jl will work 10 years later, but is it “GoldStandard” if everyone moves onto a broader Web.jl within 3 years, and Webscraper gets 5 years behind on critical security vulnerabilities? If security is of utmost importance, yanking may not go far enough, and reproducibility takes a back seat.

8

I don’t follow the logic because reproducibility is something completely different from both security and quality and quantity of content. These are all orthogonal characteristics of a workflow. I can do reproducible work when using 1 function from 1 package, and when using 1000 functions from 10 packages. Similarly, I can have a reproducible workflow that is full of vulnerabilities.

If everyone moved to Web.jl after 3 years, then I’d wager Web.jl should have no problem entering a “GoldSantard” registry.

In any case, we are now well in the point where the Julia language has 100s or 1000s of packages that are of exceptionally high quality so we are not faced with the problem of having a bunch of packages that are just starting out.

9

My argument is that it’s not orthogonal, it’s actually anticorrelated. A reproducible registry aims to keep all versions instead of pruning them like a typical software service, and that adds up over time. It’s easy to say (correctly) that most people would move onto the most used and updated tools, but that doesn’t stop the deprecated and flawed stuff from cropping up in old projects and likely LLM-involved suggestions. A higher entry barrier may reduce packages that were lower-quality in the first place, but it can’t do anything about that accumulation.

I actually remember the smaller initial debate over adding Kaimon to General, and a maintainer was very clear about minimizing gatekeeping as long as the explicit registry guidelines were met. That approach actually promotes adoption; a higher entry barrier could easily backfire by making it harder for all developers to cooperate while only mildly improving the Julia experience for the few users who can’t be bothered to ask around for package recommendations. My questioning of General recommendations wasn’t an argument against this; I’m saying the basic commitment to unconditionally sharing open-source Julia should be emphasized more strongly because not everyone who wants to reach more users has that goal. Granted, General is built to cache and revive code despite whatever developers do, but it’s still better to deter developers who want more control over their code, as is their right, and might act in defiance of an OSI license they didn’t think much of.

10

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED…

I do not see how it is technically possible to act in defiance. The MIT (and similar) license specify VERY clearly:

  • Giving the world the code puts no (zero, null) specific obligations onto the developer
  • Anyone can do practically everything with that code

As to the second point - while the developer can delete his repo (or, much more common, just abandon it without notice), the code is still out there (in the General and elsewhere), and anyone can fork and develop it further. It is not the problem, the problem is - “who?”

11

Which effectively forces anyone to jump more hoops to continue the work. Also consider what goerz said earlier:

So any Kaimon revival would a higher re-entry barrier to General, and the registry’s principles still offer a way for that to be obstructed despite what the license lets us do outside the registry. This is a good policy; nobody would participate if maintainers could easily replace developers with one of many hijackers.

12

It actually is though. A registry has nothing to do with a “software service” of a company. It’s an archive. The reproducibility comes from the Pkg system build on top and specifically the Manifest.toml file.

I can reproduce my 5 years old project currently entirely because the General Registry archive exists and because Pkg.jl defines and knows how to use Manifest.toml. Not because I haven’t moved to new packages, nor because the packages I used 5 years ago were ‘perfect’.

Reproducibility has nothing to do with people moving on to different versions of software in the present.

Reproducibility also has nothing to do with LLM suggestions or flawed software.

It’s about doing what has occurred in the past but also in the present in the exact same way. If it worked at some point in the past, and you have a very detailed record of exactly how it worked, you should be able to do it also in the present. That’s what reproducibility is about, and General Registry does a damn good job about it. All I’m asking is a registry that does this, and has high entry barrier.

13

Continuing the work a very theoretical possibility anyway. Historically, some projects may have been taken over after being abandoned. There may be some chance of it for projects which are 1. Not overly complex, and 2. Are domain-oriented, so there is a chance some PhD student needs it for their research.

For a tooling project however, the only developer’s incentive is practically only their care for “their baby”. So while a chances of a general project being taken over by a new maintainer after a period of being abandoned are just low, those chances for projects like Mousetrap.jl or Tachikoma.jl are practically zero, and any discussion about “more hoops” is thus moot.

14

I think I must have seriously misread your intentions for the higher entry barrier. If a registry collecting junk and gatekeeping diamonds in the rough are acceptable risks, what is the barrier actually for and what benefits do you expect over General?

15

I would want a Curated registry, which would typically be a subset of General. Typical projects would start in General, and might be added to Curated at the behest of the curation committee, not the package author. And ideally it would be possible to remove projects from Curated (would still remain available in General).

The expected benefits would be:

  1. A signal of “this is high quality”.
  2. Hence, better discoverability of high quality projects.
  3. A better guarantee that the project is maintained.
  4. Different rules for project control: It would be made clear that control of the “curated” version always belongs to the curation committee. Sure it is typically exercised by the project maintainers, but the curation committee should always feel free to point to a fork if they disagree with the project management (and users of Curated/foo.jl would automatically stay on the fork).
  5. Tooling support to learn which dependencies are part of the Curated or General world.

Point (4) is painfully made clear here: @kahliburke controls the github repo for Kaimon.jl . It is not entirely clear for users whether their Kaimon.jl entry in their Project.toml is backstopped by the personal honor of @kahliburke or the personal honor of the general registry maintainers like e.g. @goerz . Normally this doesn’t matter, until in case of conflicts it suddenly does matter.

For the hypothetical “curated” registry, the rules would be very clear: Users of Curated/Kaimon.jl have chosen to put their trust with the curation committee over the package author. For users of General/Kaimon.jl, arguments could be made that @kahliburke owns the trust of their users, and hostile takeover of Project.toml entries is a nuclear option reserved for very gross misconduct (a threshold that we almost surely haven’t reached here).

This is not so different from the distinction between arxiv and peer reviewed journals. Sure, elsevier et al suck, but the concept of peer-reviewed arxiv-overlay journals is not bad. (a paper is backstopped not just by the author’s reputation, but also by the editorial board who have the sacred duty to override the author’s opinions on retraction)

Another real-world example are linux distros. The buck always stops at the distro maintainers. Most of the time this works very well.

Sometimes it fails, for example, many debian users famously trusted debian maintainer’s opinions on RNG seeding for openSSL over the opinions of openSSL developers.

Or julia on archlinux. Opinions differ on the use of systemlibs vs julia-specific forks/patches. I get to choose to either pacman -S julia (sometimes buggy, especially wrt GPU, but conforms to archlinux philosophy) or pacman -S juliaup (trust julialang over archlinux on how to best install julia on archlinux).

PS Edit: I misattributed the username of Kahliburke before this edit. Thanks @Mason for spotting that, and apologies to Kahliburke and rayegun for messing up the mentions.

16

Do you mean @kahliburke? I wasn’t aware of Raye controlling Kaimon but maybe I missed something.

17

Curated registries would be nice!

The good news: anybody could start such a registry. Realistically, I could see these by domain. For example, SciML could have a registry for “the best packages for numerical simulation”. Or the JuMP community one for “the best packages for optimization” (not just the packages from those orgs, but the wider ecosystem). Whether there’ll ever be a “curated General”, focusing on packages that have been formally reviewed is doubtful. There isn’t really any “staff” or volunteer bandwidth to do such reviews.

One point to be clear on: The current General registry is, and will continue to be “lightly curated”. It will probably move to “slightly more curated” over time (I really think we should require tests, for example), but no level of quality is guaranteed, and package quality will fluctuate and depend on the availability of community volunteers. The General registry will always be “permissive”, in the sense that any package with an appropriate name and an easily achievable minimum level of quality will be able to register in General, without restrictions or gate keeping. But the General registry is not a complete “free for all”; it is less permissive than e.g. PyPI, but far more permissive than e.g. CRAN.

Maybe a better idea (as has been discussed before) would be to have tags. E.g., “JOSS reviewed”, or “Won community prize at JuliaCon”. That does seem more feasible than setting up an entirely new registry with a curation team.

Regarding ‘control of the “curated” version always belongs to the curation committee’, I would point out that this is true by definition for any registry, by virtue of registries being git repositories. Whoever has commit access to the git repo controls the packages and versions in that registry. For the General registry, that’s the Julia core developers who originally set up that registry, and the people that have since been voted in by existing registry maintainers to become new registry maintainers. Importantly, there’s also an AutoMerge bot that has commit access, and this is how most registrations are added. At the end of the day, package maintainers have to trust registry maintainers to manage the registration of their package. For the General registry in particular, that’s a trust that the maintainers of General act in the interest of the broad community, without excessive gate keeping. I don’t think it’s something to particularly worry about, but still: package maintainers do not “own” the registration of their packages in the General registry, or any registry they haven’t set up themselves, in the strictest sense.

I could realistically see something happening at the other end of the spectrum: It would be nice to have an official “test registry”, similar to how Test PyPI relates to the normal Python package index. This would have the same setup as the General registry, with the same AutoMerge bot, except that a package submitter could always add a comment to override the bot. That is, all installable registrations would be guaranteed to automerge. The point of such a registry would mainly be to test whether a submission is going to pass through the automerge of the full General registry, to enable the easy development of multiple interdependent packages that are not quite ready yet to be submitted to the full General, and to be able to set up all development workflows around “making releases” before the package is ready for General. All of those things would make it much more realistic to make “don’t register in General until you’re ready for 1.0” a possible social norm.

Somebody would have to set up such a Test Registry, though, so, no promises that anything like that will ever happen.

P.S.: it may not be obvious, but the same package can be in multiple registries. The only restriction is that all registries have to associate the same tree hash with the same version number for the same package uuid