Say, I was wondering when does it make sense to register a new package with the Julia registry?
I am developing a set of packages for my own research. Now the paper is not released yet and the development is still on-going, so I have not published to the registry yet. YET, I keep running into small hiccups during development with messages like Expected "NewPackage" to be registered, and such when trying to import or work with private packages. I can dev the private packages, but still hit random errors like this and it takes a while to find workarounds.
Registering to the main registry would resolve these issues, but as I said, I was not sure if it was bad practice to register packages that are still in development?
At the same time, what is a good alternative. There are different things like setting up a private registry, etc. But the challenge there is that there are no videos on some of these advanced topics, and the documentation is a little difficult to follow–not meant to be a criticism, just an observation.
Well, it is definitely NOT bad to register packages that are still in development… Most packages are still in development. Register it, if you think it might be useful for others and if you are committed to maintain it for at least some time…
This is not quite as difficult to set up. I find it very useful. As far as I remember, I encountered a bug while creating a new registry which I fixed in my own fork of it GitHub - GHTaarn/LocalRegistry.jl at GHTaarn (unfortunately I have not gotten around to submitting a PR yet).
To be registered, it should be at a “minimum viable package” stage (which is what a v0.1 release typically indicates). This means it should have some non-trivial functionality. It also should have some tests, and very importantly, some form of documentation. This doesn’t mean that a package needs a full Documenter-based website: for simple or early-stage packages, putting the documentation in the README if usually sufficient. It should include a description of what the purpose of the package is, and a basic usage example. Without any such documentation, the package would not be usable for a general audience, which is a prerequisite for registration.
I would definitely encourage you to register before you submit a paper (or even preprint) about your package. This ensures especially that there are no problems with the package name. It has happened more than once that someone tried to register with a package name that doesn’t meet the guidelines, and then had to argue for an exception based on “we already published a paper with this name”. That’s a situation you should absolutely avoid.
If the package is not yet ready for a public v0.1 release, but you want to avoid issues with "Expected <package> to be registered", I would very much recommend a LocalRegistry. If you develop multiple unregistered packages that depend on each other, putting them in a LocalRegistery is almost unavoidable.
You definitely don’t need to wait until your package has reached v1.0-level maturity, where all the functionality has been implemented and is fully documented. Most packages are still “work in progress”.
@ufechner7 thanks for the response. Yes, this makes sense. I think I was setting too high a standard, but seems like the consensus is quick release of basic functionality is perfectly fine.
@goerz You raised some very good points that I had not thought about, such as the whole naming issue. That is very helpful. I understand what you mean about some minimal viable functionality, which is good.
I suppose I have to practice more with the Local Registry stuff. I have read through the docs a while back, but still found it a bit confusing. I was worried I would mess up my julia implementation if I make some errors. But I suppose I will have to just try and write some simple toy packages and experiment with a Local Registry to see how it works.
@GHTaarn Yes, I think I will have to practice using the Local Registry stuff a bit more. I read through the docs and experimented with it a while back, but it probably seems like the best method for my use cases. I just have to write some simple demos and get the hang of it. Thanks for the encouragement.
Yes, good idea to experiment with it with some play packages and registries. I just pushed some changes to the documentation to my fork GitHub - GHTaarn/LocalRegistry.jl at GHTaarn about creating new registries which I feel explains things better.
I don’t think one needs to rush to register a package. I use packages directly from Github, private and public. No problem. Only when I deem the package to be (i) usable, and (ii) useful to others, I register it.
Oh I see. Yes, that tutorial is really very helpful. I see how this is supposed to work now. I really want to record some sort of video of these things. I imagine that many other people would love to see more hands-on demos of this package :). Of course time commitments are real and always a struggle. I will work through the demo though and try to understand how the system works.
Since Pluto’s built-in package manager does not allow for using GitHub links easily, it is indeed much easier for beginners if the package is registered and I just share them a single Pluto file.
I agree with your point. Pluto is so very useful. I have not tested this yet, but I imagine that a Local Registry would also work with Pluto? @GunnarFarneback would you know if Local Registry works with Pluto–I imagine that it would?
So of course users come with different levels of expertise. If I have a user that is not very proficient in Julia, then trying to set that user up with all sorts of exotic customizations to load packages is really time consuming and fragile. In that case, just registering with the main Julia authority makes more sense.
If I have a moderately saavy user, then setting up a Local Registry on his/her machine would probably be easy and low complication. I just need to setup the registry once, and the user should not need to know whether the package is coming from the main Julia Registry or the Local Registry. And when the package is released publicly, I would just remove the Local Registry entry and create the identical Julia public registry entry. And the user would not see any difference. I might just need to adjust the using statement in the notebook to match the new public registry entry.
Just to be a little more precise about it, as @goerz said, we want to register our package names early so that we know the names are valid. So the developer would create a public registry entry that points to a TBD or “in development” branch. In our local registry we can just name our package something like “PackageName_draft”. Then we would enter the PackageName_draft info in the Local Registry. When we are ready to make the package public, we can update the public registry to the main branch, and remove the PackageName_draft entry from the local repository. And then we just update the using statement in the Pluto notebook using PackageName_draft → using PackageName. That should be it.
You can have the same package in multiple registries (e.g., a LocalRegistry and General). So you don’t need to rename packages when they’re ready for General. You can even continue using both registries in parallel (and use the LocalRegistry for pre-releases): Different registries can hold different versions of a package. All that’s required is that if the same version is in multiple registries, those registries must have the same tree-hash for that version.
If you’re not quite ready to submit to General yet but want to make sure that the package name is OK, the automatic checks are done by RegistryCI, which you can also run locally. See, for example, the instructions for the Name similarity distance check. That check in particular tends to have a lot of false positives, and those are usually merged manually without any problem. Other rules like using CamelCase and not having a Julia prefix are more strict. There are also “rules” the bot doesn’t flag, such as package names that are too generic. So it’s definitely a good idea to either register or just ask on Slack what people think about your proposed package name before you register.