Can we have pre-release versions in the registry? (v2)

[MilesCranmer]

TLDR:

• Problem: there is no way to register a pre-release of a package, which causes too much friction for potential early testers
• Goal: the general registry should allow pre-release candidates along the specification of SemVer 2.0 (e.g., -alpha, -beta, -rc with integer suffix for multiple pre-releases)
• Why?: a lower-friction pre-release workflow will encourage more users to participate in early testing of libraries

---

I think that pre-releases ought to be tracked by the registry. There have been various other discussions on this (e.g., 1, 2) though they are a bit outdated.

Here is my current situation:

1. I want some users to be able to test out the latest version of SymbolicRegression.jl and PySR before they are released to the broader community. (these are on branches at the moment)
2. That version of SR depends on multiple packages (DynamicExpressions.jl, DynamicDiff.jl, BorrowChecker.jl) which are also in an experimental phase.

A workaround could be to tell people to run something like

pkg> add SymbolicRegression#v1.12.0-beta1 DynamicExpressions#v2.0.0-beta3 DynamicDiff#v0.3.0-beta2 BorrowChecker#v0.3.0-rc1

Note the differing -beta1, -beta3, -beta2, -rc1 tags which much be specified explicitly.

I consider this to be too much friction. I want to encourage users to beta-test my software, and have it be easy. Mature software libraries often make this very easy to do, even for users who know nothing about coding. e.g., I just switched to the new Adobe Photoshop beta in CreativeCloud by clicking this high-visibility button:

This it effortless for non-technical users, who can then give feedback on early features and generate crash reports. For me, some PySR users do not know what a git tag is, but their testing and feedback is still quite valuable!

In other package registries like crates.io or pypi.org or conda-forge – alpha, beta, and release-candidate releases are supported. There are even CLI flags for automatically selecting these (e.g., pip install --pre; cargo add --prerelease). Of course you could always look up the right git tag for it, but that’s just another barrier to a user doing free labor for me by testing my package in a pre-release state. Which I really want to encourage.

If I release -rc2 for just DynamicExpressions.jl, I need to instruct users explicitly to re-run the above command with one small change each time a dependency updates. This quickly becomes cumbersome, and lowers the effectiveness of their feedback.

Julia itself clearly recognises the importance of making it easy for the community to experiment with pre-release versions (just downloaded 1.12.0-beta4 to do my part ). So I think we just need the option for the general registry to lower friction for beta testing of packages.

If there were pre-releases, we could also potentially have an option

Pkg.test(prerelease=true)

Which would default to pre-release versions of packages (for any that have been added to the general registry) – to help generate feedback on early versions of dependencies.

---

A few tangential notes:

1. One recent update to Pkg is some new Project.toml syntax ([sources]) that lets you specify custom sources like:

   PackageB = {url = "https://github.com/username/PackageB.jl", rev = "v1.0.0-rc1"}
   

   This would be a great solution to this, however, Pkg does not recursively follow sources. So you have to set all the URLs at the top-level repository you want someone to test, and update the Project.toml (and update the git tag) whenever a dependency releases a new beta version. So, it’s doable, but too much friction.

2. A “monorepo” is a good option for many, but not something I can use because these libraries are fundamentally for different things and have different uses. But they often participate in the same release cycle due to the requirement of new features in dependencies.

[giordano]

Small note: you probably want to use the -alpha.N label, rather than -alphaN (and similarly for -beta, -rc, etc):

julia> v"1.0.0-beta12" > v"1.0.0-beta2"
false

julia> v"1.0.0-beta.12" > v"1.0.0-beta.2"
true

julia> dump(v"1.0.0-beta.12")
VersionNumber
  major: UInt32 0x00000001
  minor: UInt32 0x00000000
  patch: UInt32 0x00000000
  prerelease: Tuple{String, UInt64}
    1: String "beta"
    2: UInt64 0x000000000000000c
  build: Tuple{} ()

julia> dump(v"1.0.0-beta12")
VersionNumber
  major: UInt32 0x00000001
  minor: UInt32 0x00000000
  patch: UInt32 0x00000000
  prerelease: Tuple{String}
    1: String "beta12"
  build: Tuple{} ()

[MilesCranmer]

Thx. Any idea why Julia itself is not using that?

julia> VERSION
v"1.12.0-beta4"

[giordano]

I guess that’s a question for the release managers.

[MilesCranmer]

Will post an issue!

Edit: Julia prerelease tags are parsed as lexical strings, potentially causing ordering issues · Issue #58734 · JuliaLang/julia · GitHub

[ericphanson]

When should Pkg resolve packages to pre-release versions?

[MilesCranmer]

I think other package managers have pretty reasonable rules about this, so I would just go with those. The ones that I know (Cargo & PyPI) do something like the following:

Say you have tested -beta.2 and verified it works as a prerelease version. You would write:

[compat]
MyPackage = "1.0.0-beta.2"

This should permit the following versions:

• 1.0.0-alpha.5
• 1.0.0-beta.2
• 1.0.0-beta.3
• 1.0.0-rc.1
• 1.0.0
• 1.1.0-alpha.1
• 1.1.0-beta.1
• 1.1.0-beta.2
• 1.1.0-rc.1
• 1.1.0
• 1.1.1-alpha.1
• 1.1.1
• 1.2.0
• 2.0.0-alpha.1
• 2.0.0

So most of this should just work because of the existing VersionNumber ordering. The one thing that needs changing is filtering pre-releases on a different major.minor.patch when resolving. But hopefully that’s not too bad to set up?

(I think this filter is quite reasonable: “just because I know the prereleases 1.0.0-* are viable after beta.2, does not mean I want to opt-in to 1.1.0-alpha.1 as well”)

Similarly, if your compat is simply 1.0.0, you would just filter all prereleases. It would be like normal.

---

Another thing other package managers have is a --pre or --prerelease CLI flag, which permits absolutely every pre-release within the compat bounds. Those would turn off the filtering described above. If these are not specified, then the pre-releases are basically invisible unless explicitly asked for.

However, this would be a 2nd order feature. I think simply permitting prereleases in the [compat] bounds (with the above resolver - or even requiring exact = resolution) is much higher priority.

[ericphanson]

I think that makes sense for compat bounds. What if there’s no bound? Would the rule be no rcs or any rcs?

[MilesCranmer]

No rcs.

[GunnarFarneback]

For a different approach, I think you can get a lot of the functionality you want, already today, by creating a pre-release registry for your packages and ask your testers to add that registry. You would register your pre-releases yourself with LocalRegistry rather than Registrator (which you probably will find is easier anyway).

I’m not sure you can convince Pkg to do fine-grained compat on a variety of pre-release tags, but if you mostly want your testers to be able to update all involved packages to your latest pre-release versions, I believe that should work.

[MilesCranmer]

Thanks. That makes sense, though it’s still too much friction for me.

@GunnarFarneback

I’m not sure you can convince Pkg to do fine-grained compat

Just curious: why is that? Every other package manager I use seems capable of it.

[nsajko]

A drawback, if this proposal gets implemented, is that it seems like it would exacerbate the preexisting issue where lots of packages lack a stable release. Previously this was restricted to making the major version be zero, but if this gets implemented, I suppose maintainers might become even more skittish around cutting a stable release.

[nsajko]

@MilesCranmer

Julia itself clearly recognises the importance of making it easy for the community to experiment with pre-release versions (just downloaded 1.12.0-beta4 to do my part ).

This certainly seems like a “do as I say, not as I do” situation. I suppose the distinction is that Julia is a product on its own, while the packages in the General Registry are usually meant as libraries, not as “apps”?

[aplavin]

@nsajko

it would exacerbate the preexisting issue where lots of packages lack a stable release

Is it actually an issue with lots of packages? There are indeed some packages that make breaking changes often, but there are only few of these. In my experience, most Julia packages tend to be conservative and stay stable for years.
Do you have specific examples in mind?

@nsajko

Previously this was restricted to making the major version be zero

Package stability doesn’t really depend on version number. A useful stability metric could be something like “typical time between breaking releases” – what would be quite interesting to evaluate across the Julia ecosystem! Whether the release is 0.12 → 0.13, or 12 → 13, the result is the same.

[aplavin]

@GunnarFarneback

creating a pre-release registry for your packages and ask your testers to add that registry

This can indeed be pretty convenient for both devs (LocalRegistry is more straightforward than General) and users (need a single registry add)!
Probably the only issue is version numbering. If you release 2.0.0 on the testing registry, you don’t want to release a General 2.0.0 with a different content afterwards. So, the latter should be 2.0.x I guess?

[e3c6]

What about dependencies of the package which are themselves beta?

Suppose you specify:

[compat]
MyPackage = "2.0.0-beta.2"

and MyPackage depends on DepA which as registered versions 1.0.0 and 2.0.0-beta.1.

Which DepA should we resolve to?

1. If v2.0.0-beta.2 of MyPackage requires version 2 of DepA, I think it’s pretty clear that the resolver should install 2.0.0-beta.1 of DepA.
2. If v2.0.0-beta.2 of MyPackage is compatible with both versions 1 and 2 of DepA, what should we do? I think installing v1.0.0 of DepA is sensible in this case, though probably as a developer you want to get v2.0.0 of DepA tested.
3. Lastly, if the user instead requests a released version of MyPackage, by writing in the compat MyPackage = "1.0.0", then is clear all beta versions of dependencies such as DepA should be ignored.

So it seems to me only situation 2. above is ambiguous, and it is not clear to me what’s the best approach in that case.

[MilesCranmer]

@nsajko

I suppose the distinction is that Julia is a product on its own, while the packages in the General Registry are usually meant as libraries, not as “apps”?

Hm, I wonder if this is a Julia cultural thing? Or perhaps about ecosystem maturity? I have some Julia-based tools that I consider more “app-like” that I want to put out some pre-releases for. Should Julia apps be released in a different package manager than Pkg; maybe even on conda-forge?

Then again, I think it would be a pity if Julia ended up with multiple competing package managers like Python. So not sure.

[MilesCranmer]

@aplavin

pretty convenient for both devs (LocalRegistry is more straightforward than General) and users (need a single registry add)!

Just to add some colour to what I mean when I say I want to lower friction for ‘users’: there is a group of users of PySR and SymbolicRegression.jl whose only prior coding experience is Excel formulae. They may not know what “compilation” is. Still, though, they are capable of emailing helpful bug reports. I’m eager to make it more straightforward for users – even at that level – to participate in beta testing.

(Though maybe not so eager I am up for maintaining custom installers and registry for a manual workaround. Though it seems like one could which is nice!)

[MilesCranmer]

I would just do what both Cargo (rust) and PyPI (python) do (and maybe conda as wel?l)

1. This should be a resolver error. Unless you specify --pre(release) when installing which will override the constraints.
2. Yeah. Will install 1.0.0 unless you have the prerelease flag - in which case you get the 2.0.0 beta.
3. Yes

[aplavin]

@MilesCranmer

I’m eager to make it more straightforward for users – even at that level – to participate in beta testing.

For Julia users it is very straightforward: ]registry add https://your/registry/url in the familiar Pkg prompt. Not sure how clear it is to julia-through-python users, but it’s still a single command line call to opt into beta testing for your packages.

Note: I agree that official support for pre-release versions would be conceptually cleaner. Still, in the meantime a custom registry seems like a nice solution.