Pre-release versions of packages

Internals & Design
1

My understanding is that since Pkg uses SemVer, pre-release versions like 2.0.0-rc.1 and similarly alpha and beta would be technically allowed for package versions.

What I would like to avoid is unsuspecting users upgraded from 1.4.0 to 2.0.0-rc.1 next time they Pkg.update(). Are there protections against this, or this is something one should avoid at this point by not releasing pre-release versions? My reading of Pkg.UpgradeLevel is that patch versions are nothing special and so the above update would just happen.

5 Likes
2

I guess one answer is: don’t register pre-release versions. Beyond that I guess we’d need a way for users to opt into getting pre-releases. You can always ask users to explicitly install pre-releases for testing.

4 Likes
3

Whether it’s a pre-release or not, automatically installing a new major version can break things.

Would it make sense to have a minor_update function (as an alias for update with level = UPLEVEL_MINOR)?

This would be a safer choice for people who don’t want to deal with breaking changes in their semver-compliant dependencies.

1 Like
4

Actually no. Version identifiers are only given as x.y.z and the resolver and the registry only understands that format.

3 Likes
5

Is registrator/tag bot smart about pre-releases / releases? It would be great to have the functionality to opt into early releases.

6

Then perhaps I misunderstood the docs:

the Julia package manager respects semantic versioning (semver)

because semver explicitly allows pre-release versions.

I understand that for a specific package, one can always install a branch or a tagged commit, so users can try out pre-release versions. But for a suite of packages, it would be easier to pre-release all of the dependencies, too.

1 Like
7

You’re allowed to tag pre-release versions, you just can’t register them currently.

8

So, just to recap, is the current viable solution to make an announcement like this?

Package Foo is ready nearing a new, major release, with breaking API changes and tons of improvements. If you want to test it before it is released, do

pkg> add Bar#master Foo#master

to install all dependencies correctly.

and not worry about pre-release tags since they go through the registry, so would have to be pulled manually?

1 Like
9

To clarify: git allows you to tag things, regardless of whether it’s registered or not. You can also make a release on GitHub regardless of whether it is registered or not. The registration machinery doesn’t currently allow you to register pre-release versions, but it potentially could if we decide it’s a good idea. Have you tried tagging a prerelease and then using #1.2.4-alpha in place of the branch name? That might already “just work”.

1 Like
10

Some coworkers and I were having a discussion about this very same topic just last week, so this question is timely. The simple git tag and then add MyPackage#1.2.4-alpha should work fine for a single standalone package.

But if MyPackage version 1.2.4-alpha depends on a pre-release of MySupportPackage (say, 0.8-alpha), or possibly an even larger set of pre-releases, then users would need to do

add MySupportPackage#0.8-alpha
add MyPackage#1.2.4-alpha

similar to what @Tamas_Papp mentioned above. This is a fine workaround to expect testers of pre-releases, but it would be awesome if Registrator and Pkg could handle pre-releases.

11

For this to happen the following would need to be done:

  1. Fixup the resolver to handle it. https://github.com/JuliaLang/Pkg.jl/blob/89603411622bc725f9f4b8f4fc94b0ac8772982d/src/resolve/VersionWeights.jl#L7-L13 etc.
  1. Fixup VersionRange in Pkg to handle it.
  2. Fixup Registrator.jl to handle it.

For the people that tend to work on Pkg, there are some more pressing issues to work on so this would likely need to be contributed by someone who has a strong desire for it.

7 Likes
12

Given that the potential early adopters of beta versions of packages are probably power users, I think that simply asking them to track #master of various packages is fine.

An API for this in Pkg would probably be equally or more complicated (does the user want to track pre-releases of all packages? if not, then which? yet another config, etc).

I was just asking for clarification, and now I understand things better. Thanks for all the answers.

13

There would have to be some kind of rule to avoid complexity. For example, “avoid pre-releases unless they are the only way to satisfy version constraints”. So in the examples above, you would never install any pre-releases unless

  • the user specifically added (directly or indirectly) package at a pre-release OR
  • some added or installed package requires a dependency at a pre-release version

I don’t think that’s too complex to wrap our heads around and would ease the “please test my package before I release it so I don’t break your stuff” problem.

Thanks @kristoffer.carlsson for outlining what changes would be needed to make this happen – I totally understand about developer priorities. Perhaps one of us will take a look at these needed changes and see if it’s something we can take a bite out of.

1 Like
14

This is easy to automate if MyPackage uses test/Manifest.toml. The idea is that, if your direct dependency needs a pre-release version of the indirect dependencies, your direct dependency already must have the indirect dependencies with appropriate versions in its test/Manifest.toml. So, you can use MyPackage’s test/Manifest.toml to figure out that you need MySupportPackage#0.8-alpha. I’m doing something similar in Rogue.add.

1 Like
15

For people finding this via search but wondering whether the information is stale, this is still accurate w.r.t. the level of support currently available.

2 Likes
16

This should be easier with the new Pkg.jl functionality: 10. Project.toml and Manifest.toml · Pkg.jl.

For cases where you want to have release candidates for a bunch of interconnected packages (in separate repos), I think could release a git tag v1.0.0-rc1 that defines a Project.toml with:

name = "PackageA"
uuid = "..."
authors = ["..."]
version = "1.0.0"

[deps]
PackageB = "..."

[compat]
# PackageB = "1"  # unreleased

[sources]
PackageB = {url = "https://github.com/username/PackageB.jl", rev = "v1.0.0-rc1"}

And then a user would theoretically only need to do

pkg> add PackageA@v1.0.0-rc1

and get all the release candidate dependency as well.

However, I haven’t tried this. I’m not sure if [sources] will propagate from the PackageA repository, bringing the source for PackageB. @kristoffer.carlsson would you happen to know?

18

Fixed

1 Like
19

I don’t think that’s how it works. It doesn’t apply recursively, pretty sure you have to add [sources] to each Project.toml that has non-registered dependencies.

20

Thanks for correction. Yeah it looks like this is not possible.

FYI I’m going to start a new thread to avoid keeping this one alive, since it’s a bit out of date.

2 Likes
21

There are two issues with prerelease versions:

  1. The current resolver can’t really handle them. I have a new resolver implemented that fixes this but I haven’t replaced it yet.
  2. The syntax the registry uses for deps and compat stuff doesn’t really work with pre-releases. As long as pre-releases and releases have the same deps and compat then it should be fine but this doesn’t seem possible to guarantee.
4 Likes