# The present and the future of package registration

**URL:** <https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890>\
**Category:** Package Management\
**Created:** [June 5, 2023, 1:40pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890 "2023-06-05T13:40:11Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [June 7, 2023, 3:43pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/61 "2023-06-07T15:43:01Z")

</div>

There are 300 people in the channel, that would have to be a hell of a vacation! It seems to me that a check for “this package name might be a bit too generic” is hard to automate, so getting every registration in front of loads of people’s eyeballs seems to me the best feasible solution.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [June 7, 2023, 3:47pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/62 "2023-06-07T15:47:18Z")

</div>

As with most channels, the participation rate is far, far lower. I suspect most are either inactive on Slack or have it muted.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [June 7, 2023, 3:51pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/63 "2023-06-07T15:51:19Z")

</div>

I didn’t mean to imply that all of those are active, but in my experience every package for which I think “oh, that seems potentially problematic” already has a comment when I click through to GitHub. Hard to say how many people click through on the links, but I’d be surprised if it was less than 20 (which should give us holiday cover!)

---

<div class="post-metadata">

**Author:** ![goerz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerz/32/3269_2.png) [@goerz](https://discourse.julialang.org/u/goerz)\
**Post date:** [June 7, 2023, 4:08pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/64 "2023-06-07T16:08:18Z")

</div>

Yeah, I don’t know how to solve the problem. I guess the options are:

1. We just organically have enough people keeping an eye on the new-packages-feed so that things don’t slip through. I, personally, like to check on it, and I’ll definitely leave comments, but I have no idea how many other people really keep up with the feed
2. We just live with inconsistent enforcement. Not ideal, but it’s been working in the past, and I don’t know how much bigger the problem will get in the future
3. There’s a (paid?) package community manager whose job it is to keep an eye on the feed, flag package names that might need some discussion, and generally helps people with the package registration process if needed. Probably the ideal solution, but who’s going to hire that person 🙂
4. We disable automatic merging of new packages. This guarantees that nothing will slip through the crack, but I don’t think it’s sustainable (except maybe with the package community manager). I don’t think anyone really wants this level of overhead.

Actually, I think it’s slightly problematic that the registry maintainers seem to only be reachable via Slack (which is why I [recently](https://github.com/JuliaRegistries/RegistryCI.jl/pull/498) made the pointer to `#pgk-registration` in the Registry CI more explicit). There have been instances where people trying to register a new package (somewhat justifiably) complained that they had to jump through multiple communication channels to finally get their package merged. It would be much nicer if there was a group of people monitoring new registrations to General _on Github_. If someone comments on their PR on why they like to stick to their name despite the name similarity, someone should react to that without further prompting, ideally.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [June 7, 2023, 4:17pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/65 "2023-06-07T16:17:47Z")

</div>

Good point about GitHub. Having some sort of automation that mentions a GH team and/or adds relevant labels when a package is flagged for discussion (maybe with variation for appealing the automerge decision vs identifying packages that need more scrutiny) would be great too.

---

<div class="post-metadata">

**Author:** ![mihalybaci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mihalybaci/32/13528_2.png) [@mihalybaci](https://discourse.julialang.org/u/mihalybaci)\
**Post date:** [June 7, 2023, 4:43pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/66 "2023-06-07T16:43:40Z")

</div>

I recall a different thread on this topic, which escapes me now unfortunately since I’ve made this point before. But even if packages could be added by org, it still seems worthwhile to disallow packages with the same name. A perfect example is to go to [JuliaHub.com](http://JuliaHub.com) and search “unitful”, there are many “UnifulXX” packages (Unitful, UnitfulAstro, UnitfulUS, UnitfulMoles, etc.). Now imagine that doing that search but where every single one is `Unitful.jl`, some in big orgs (`JuliaAstro/Untiful.jl`) and some not (`gituser12345/Unitful.jl`). That seems **way** worse for discoverability to me than the current _status quo_.

It also would be reasonable to expect someone to need more than one units package anyway, and I don’t really like the idea of doing

```julia
using JuliaAstro/Unitful as UnitfulAstro
using PainterQubits/Unitful as UnitfulUS

```

Because it doesn’t buy me anything.

So, adding packages with an org/user could be helpful, but it doesn’t strictly solve everything, and I would imagine a lot of Discourse posts beginning with `which version of Untiful should I use?`

(I’m not picking on `Unitful`-packages here, I love them all and that they’re all named differently!)

(edit: @lmiq I’m also not saying that’s what your point was, yours was just the first post to mention the idea of adding orgs to packageds names so I hit `reply`.)

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 7, 2023, 5:22pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/67 "2023-06-07T17:22:25Z")

</div>

“Which version should I use?” comes up either way. Either “orgX/Unitful”, “orgY/Unitful” – or “UnitfulX”, “UnitfulY”, etc. If anything, belonging to a “well known” organisation can provide a degree of authority straight in the name.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 7, 2023, 5:29pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/68 "2023-06-07T17:29:17Z")

</div>

Maybe I’m the ultra-liberal in the room, but for the life of me I can’t understand why there should be limits on how I name my package, or how I document, or test my package? As long as I’m not doing anything offensive, I should be allowed to register it - and the Julia users can decide if they want to use it or not.

The very basic argument is that registration checks are practically useless:

- the entire codebase can be changed after the registration
- useless tests can be easily added

In addition, it can lead to subjective decisions. As in the case of “generic” names - when is a package generic enough to be allowed a generic name?

Finally, naming a package is also a branding decision, and frankly, these descriptive name guidelines are boring as hell.

* * *

Edit 1: please don’t take the discussion in the direction of the usefulness of tests and documentation, I fully agree on their value.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [June 7, 2023, 5:37pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/69 "2023-06-07T17:37:14Z")

</div>

You’ve been told the reasons a few times now - maybe you don’t agree with them, but surely you should understand why they exist.

And you seem to conflate a little extra friction with being prevented altogether.

Something doesn’t need to be foolproof or ironclad to be useful, as well.

For better or worse, General is a loosely curated registry, and the Julia community appreciates that it is what it is.

Maybe there should be a free-for-all registry, as well, and/or namespaces, and/or better ecosystem to support multiple registries. I think it’d be useful to have that. But General being lightly curated is a benefit.

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [June 7, 2023, 5:37pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/70 "2023-06-07T17:37:31Z")

</div>

There are soon 10000 registered packages. Every package added increases the entropy of the whole system (yes, sure I know, in many cases the package is also useful for something and somebody).

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 7, 2023, 5:41pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/71 "2023-06-07T17:41:06Z")

</div>

> [@mihalybaci](#):
>
> ```julia
> using JuliaAstro/Unitful as UnitfulAstro
> using PainterQubits/Unitful as UnitfulUS
> 
> ```
> 
> Because it doesn’t buy me anything.

Certainly there won´t be a one-size-fits-all solution. But this example for me provides exactly the contrary argument: You would only use `as ....` if you wanted to qualify the names of the functions of each package, which mean that they _are not_ meant to interoperate. Which is not the case, at all, here. The package are actually meant to work together as extensions of one another, and having the same name under different “physics-field” organizations would be quite indicative of that.

Anyway, I certainly would prefer this from a possible proliferation of custom registries, which would have these and many other downsides.

What I like most about “under-organization” packages is the possibility of the organizations to give more clear indications of the interoperability of its packages, taking somewhat the place of the “mega-packages” that exist in other languages but are unnatural to Julia. Yes, all that can happen already, but would/could be more explicit with a tree-based package organization.

---

<div class="post-metadata">

**Author:** ![goerz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerz/32/3269_2.png) [@goerz](https://discourse.julialang.org/u/goerz)\
**Post date:** [June 7, 2023, 5:51pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/72 "2023-06-07T17:51:48Z")

</div>

> [@essenciary](#):
>
> why there should be limits on how I name my package, or how I document, or test my package

There aren’t limits on packages development. There are limits on what you can _register_ in General, because General sustains the Julia community as a whole.

This is going off on a tangent, but what I really like to see is support in `Project.toml` to point to an arbitrary `LocalRegistry`. As in, “Get this package from this registry”. That would make registries for specific sub-communities or organizations with their own guidelines much more feasible. I’m not sure how explicit that is in the long-standing [Store some location info for deps in Project.toml · Issue #492 · JuliaLang/Pkg.jl · GitHub](https://github.com/JuliaLang/Pkg.jl/issues/492), so maybe I should open another issue for that specific idea.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 7, 2023, 5:54pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/73 "2023-06-07T17:54:32Z")

</div>

> [@goerz](#):
>
> This is going off on a tangent, but what I really like to see is support in `Project.toml` to point to an arbitrary `LocalRegistry`.

An what if that local registry disappears? This can lead to a new rabbit whole of irreproducible environments.

---

<div class="post-metadata">

**Author:** ![goerz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerz/32/3269_2.png) [@goerz](https://discourse.julialang.org/u/goerz)\
**Post date:** [June 7, 2023, 5:58pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/74 "2023-06-07T17:58:25Z")

</div>

Tough luck for users of that package, then, I suppose. I don’t see that as something that’s likely to happen. Registries are Github repos, so as long as GitHub is around, someone would have to go out of their way to delete a registry. Plus, people can clone/fork them!

People have unregistered packages, which they can also delete. That’s why General will continue to be the default: it comes with stronger long-term guarantees (which also includes stricter rules for what can make it into General)

I’d say if you’re a developer who chooses to put your package in some registry other than General, it’s your responsibility to make sure that registry stays available.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 7, 2023, 6:04pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/75 "2023-06-07T18:04:13Z")

</div>

Yes, ok, agreed. That’s a nice feature anyway to have in the Project.toml.

What I would not like to see is a tradeoff between distributing in General and distributing in custom registries that ended stimulating the proliferation of custom registries. That would be bad.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [June 7, 2023, 6:39pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/76 "2023-06-07T18:39:46Z")

</div>

> [@goerz](#):
>
> I’d say if you’re a developer who chooses to put your package in some registry other than General, it’s your responsibility to make sure that registry stays available.

The packages themselves don’t have the guarantees that General itself has, though - the packages could disappear just as well as a custom registry referencing them does.

Registries also aren’t stored in Project.toml or Manifest.toml right now - they’re just part of the environment. So that would need to change.

But let’s say it did - perhaps Manifest.toml just needs to store the last-known repository URL from the registry, so that it can fall back to the specific package url if the custom registry were to disappear or wasn’t explicitly loaded in the environment.

---

<div class="post-metadata">

**Author:** ![goerz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerz/32/3269_2.png) [@goerz](https://discourse.julialang.org/u/goerz)\
**Post date:** [June 7, 2023, 6:51pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/77 "2023-06-07T18:51:00Z")

</div>

> The packages themselves don’t have the guarantees that General itself has, though - the packages could disappear just as well as a custom registry referencing them does.

Good point! Although still not something I would worry about. If you really need to _guarantee_ reproducibility for regulatory reasons, you probably should archive a src-snapshot of all your dependencies.

> Registries also aren’t stored in Project.toml or Manifest.toml right now - they’re just part of the environment. So that would need to change.

That’s exactly what I was proposing

> But let’s say it did - perhaps Manifest.toml just needs to store the last-known repository URL from the registry, so that it can fall back to the specific package url if the custom registry were to disappear or wasn’t explicitly loaded in the environment.

I agree! That sounds like an excellent feature, but it should be in addition to support for registries in `Project.toml`. I mostly treat `Manifest.toml` as transient, or for reproducibility if the _exact_ versions matter. I’ll probably check in the final `Manifest.toml` for a research project with a bunch of notebooks when the project ends, but not while it’s active. The `Project.toml` should have all the information required to instantiate a project, even if it uses unregistered packages or packages in registries other than General.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 7, 2023, 7:05pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/78 "2023-06-07T19:05:04Z")

</div>

> [@goerz](#):
>
> specific package url

If the package repository is deleted that won’t exist anymore.

I mean, the way Julia treats packaging and reproducibility is somewhat dependent on the General registry. And it is one of Julia great features. Let’s not weaken it. With 10k+ packages already registered the exact list of packages is not important anymore. The integration of the packages with the dependency manager is much more important.

---

<div class="post-metadata">

**Author:** ![goerz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerz/32/3269_2.png) [@goerz](https://discourse.julialang.org/u/goerz)\
**Post date:** [June 7, 2023, 7:55pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/79 "2023-06-07T19:55:22Z")

</div>

I agree completely!

> Julia treats packaging and reproducibility is somewhat dependent on the General registry

Very true, but something I’m _slightly_ ambivalent about. I definitely agree, we should have as many packages as possible in General once they pass a minimum threshold of quality, and subject to some light-handed community guidelines. The benefits of General especially for reproducibility are huge, as you point out. The rule that packages in General can only depend on other packages in General should stay in effect, which already provides a huge network effect to ensure that General stays the default.

At the same time, I’d also want to slightly ease up on how tightly `Pkg` and General a coupled. There are huge pain points with working with completely unregistered packages, to the point where I would say any package that any other package depends on (including just for its tests) has to be registered somewhere.

That means a `LocalRegistry` is the only viable option for anyone who doesn’t want to register their package in General for any reasons like the following:

- The package just isn’t mature enough yet (but is a dependency for other unregistered packages)
- It’s a one-off that won’t be maintained going forward (outgoing PhD student packaging up their code)
- They don’t really care about integrating with the community, e.g. it’s a company or research group that just wants to do their own thing, which somehow feels like it should be their prerogative, see @essenciary’s [“liberal” view point](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/68).
- They disagree with the community standards like package naming (just another form of “don’t care about integrating with the community”, with all the drawbacks that entails). Again something I’d like to discourage, but which should ultimately be their prerogative.

`LocalRegistry` works pretty well, of course, but it’s still completely external to `Pkg`: You have to put instructions to run `pkg"registry add <repository url>"` manually in your README/company wiki, and more importantly, it complicates setting up tests/CI/Makefiles quite a bit. Again, I just think `Project.toml` should be completely self-contained and account for non-registered packages and non-standard registries. This might actually take off some pressure from General, as it might cut down on “pre-mature” registrations. That is, registrations happening primarily because it’s such a pain to work with non-registered packages (especially for newish users who haven’t gone deep enough down the rabbit whole to even know about `LocalRegistry`).

Anyway, I think we actually all agree that these might be nice features to have, as well as that we’d like to encourage a strong and vibrant community centered around the General registry, so the rest is just details 🙂

---

<div class="post-metadata">

**Author:** ![jlapeyre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlapeyre/32/4514_2.png) [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)\
**Post date:** [June 7, 2023, 8:40pm UTC](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890/80 "2023-06-07T20:40:54Z")

</div>

Yes, I had to hunt around on discourse to find how to add a registry in github CI. It’s doable, but not really easy to find. This is from someone who mostly copies and edits CI yaml files and hopes they work.

[Previous page](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890.md?page=3)

[Next page](https://discourse.julialang.org/t/the-present-and-the-future-of-package-registration/99890.md?page=5)