Should all these replies related to semver and package registration be moved to a different topic on Discourse?
1 Like
I would add that the process of registering and keeping a fine control over versions is also a learning curve. For new contributors it feels more inviting learning all this with 0.X package versions.
My sensation is that creating a barrier for registering 0.X packages may create an impression that new packages are not that welcome unless in a very mature state. I would not take that route.
Encouraging people to register 1.X packages in a positive way is a more welcoming message.
15 Likes
I can imagine the Hacker News comments now.
Yeah, julia has some cool ideas, but it’s unusable because of 1-based Semantic Versioning.
17 Likes
Yeah, I stick with zero-based versioning.
EDIT: To be serious for a moment, I don’t want to create a breaking release for no reason. But I’ll definitely consider starting at 1.0 for the next package I register.
10 Likes
Logically: starting registered packages from 1.0 would be always preferable for the reasons discussed above.
Psycho-logically: especially for a less-experienced package author, having to register a not yet quite stable package at 1.x version feels like taking too much responsibility.
May be prevent automerging if either the package version is higher than e.g. 0.3, or if the package is already used by many other registered packages.
A zero-based versioning, e.g. ImageCore (started 2015, currently 384 dependent packages - current version 0.9.4), shouldn’t be the regular case.
2 Likes
Whether 1.x versioning is a good practice or not is one thing. Whether all packages should follow the 1.x versioning is a different matter. There are always possibilities of good reasons to depart from a single standard. Enforcing following a single standard just creates too much centrality of decision making. It’s better off to let individual package developer to decide what he/she wants to do based on the specific circumstances.
Just to be very clear about my suggestion, I was referring to the automerge of new packages into the Julia General Registry. This does not preclude v0 packages from being merged into the Julia General Registry. This could still be done manually.
Updates to existing v0 packages in the registry should continue to be automerged.
Please don’t. First of all, 0.x releases indicate an experimental API. That is, as the package author, I will make changes without any consideration for backward compatibility. This is an important part of software development: A good API often only becomes apparent after extended usage. I want to absolutely push back on @oxinabox’s statement that
For open-source libraries users immediately depend on your API.
I definitely want “the public” use experimental versions of my package because their experience will inform the final design of the API that will become 1.0. But they should also know that they cannot yet rely on a stable API. I strongly believe that in most cases, v0.1.0 is the appropriate initial release of a package. “We’ll only release v1.0.0” might make sense for companies, but certainly not for smaller or academic projects.
That doesn’t necessarily preclude limiting the General registry to v1.x versions, but given the current state of Julia’s package tooling, it does not seem like a good idea. In principle, you could make General more restrictive, but I think that should mean putting actual quality control into what can be registered to make sure that the v0.1.0 label is actually justified: At the very least, the package should be fully documented and have good test coverage. I’d even say it should fulfill standards similar to that of a JOSS submission.
The problem with making General very restrictive is that (despite everyone’s claims how great Pkg is compared to, e.g., Python packaging), Julia’s support for non-registered packages is, frankly, terrible. You are almost certain to run into severe issues if you ever have something that depends on an unregistered package. I’ve moved to using a local org registry to avoid most of these problems, but even that isn’t ideal. There would be some ways to improve the situation:
- Have a lower-tier “test”/“experimental” registry that’s also available to Julia by default and has very minimal requirements for automatic registration (basically anything will be auto-merged as long as it installs and doesn’t conflict with any existing names)
- Add support for recording non-default registries in
Project.toml: this would get around having to put “Runpkg"registry add <repository url>"before instantiating this project:” in my project README (which people will miss and then be confused) - More generally, add support for specifying the location of unregistered packages in
Project.toml: Source of packages should be recorded in the manifest · Issue #378 · JuliaLang/Pkg.jl · GitHub
If Pkg can be significantly improved so that working with packages not in General actually becomes feasible, then by all means: add more quality control to General. Until then, though, all that requiring 1.x for auto-merging is going to do is to make people change the initial version of their barely functional new package to 1.0 just to make the bot happy. I don’t think that’s something anyone would like to encourage.
22 Likes
I disagree, a sentiment I expressed before here.
Stricter policies would heavily used libraries like FunctionWrappersWrappers would instead exist as undocumented code inside SciMLBase.jl, or something similar.
I prefer having the flexibility of placing that code in its own repo instead.
The library currently has 7 direct and 369 indirect dependencies.
The people I created it for obviously have no trouble using it, and I don’t have a reason to care about anything else.
I have zero intention to publish in JOSS.
I don’t see why a package like this should be banned, forcing further centralization into aggregate packages.
3 Likes
Would it really hurt to add a paragraph or two of documentation to the README?
3 Likes
If we start requiring 1.0.0 for merging into General, we weaken the signal of version numbers >=1 because our ecosystem has just been built around starting at 0.1. It doesn’t really matter though, I think the “signaling” part is very overrated and other cues for maturity are much more relevant. Also, even breaking releases aren’t all the same. In Makie, we have made small and big breaking releases, we always tag breaking if it’s technically so but the effort for users to keep up can vary a lot. I don’t like holding development back or bundling such updates over a longer time just to reduce the number of breaking releases, I think the amount that Makie breaks will only reduce the more problems we work out, not because we decide “no more breakage” and move to 1.0 for some psychological reason. The bugs and problems would still be there.
5 Likes
My view on this as someone who depends on Julia packages in a professional development/production environment:
- If we were only allowed to use packages past 1.x.y, we wouldn‘t get anything done. Many cornerstones of the ecosystem are fairly mature, but are still in version 0.x.y. I would like to see them move to v1 while making those long overdue breaking changes that would also remove some technical debt, but I understand that everyone is busy with more important stuff or that sometimes it is hard to achieve consensus about these matters.
- To prevent things from constantly breaking in development, we use compat bounds that exclude breaking releases. If everyone uses semver properly, this will keep things fairly stable. Most of the time when something breaks, it is accidental, and there is little that can be done to prevent that except better tests and more people reviewing. People also depend on internals of other packages, which accounts for many bugs and could be avoided at the cost of perhaps slower development in some cases.
- Because of compat, moving to breaking releases is a somewhat conscious effort. I think that‘s good, even though it is not the most exciting type of work.
- In production, freezing everything (Julia version, manifest) is almost mandatory because of those accidental bugs creeping in otherwise.
I don‘t think that changing the registry policy would solve any of the above problems, but it would create additional friction for developers. Registering for me is about ease of distribution and installation, and encourages experimentation and testing. One can always restrict a package version to stay wherever it is currently, until it creates an unresolvable constraint, and this can make even experimental packages stable. My bigger concern is that very good packages get abandoned for various reasons and the ecosystem evolves past them to the point where they become unusable.
13 Likes
As far as I understand, it’s impossible that a registered package depends on non-registered packages.
Would it be possible to relax the rule about 1.0 being a breaking change? I.e. from now on it could signal people are somehow happy with their package, and can then go from 0.x to 1.0 without a breaking change. And then in case you need a breaking change go straight to 2.0?
3 Likes
You wouldn’t go straight to 2.0, you’d first go to 0.(x+1) and then immediately tag 1.0 after that with the same code, like what we did with Julia 0.7 and 1.0 releasing simultaneously.
2 Likes
Wild suggestion: 1.0 must not be breaking, but rather it must be compatible with 0.x for the largest x on the registry. A compat bound ^1 would include the range 1.x to 0.1, so packages would update to 1.0 transparently. Packages depending on 0.z can specify that explicitly.
5 Likes
This. A hundred times Yes.
1 Like
The General registry has this policy, for very good reasons. It’s not a requirement from the package manager though and other registries could do things differently if they want. However, for most private registries it would still be recommended to have all dependencies in either the own registry or in General.
" used by" - in passive.
Meaning e.g. if many other registered packages already depend on one’s package, currently e.g. in version 0.3.4, for a year or two, the package author should consider moving to version 1.0, even if (s)he doesn’t consider the API perfectly stable yet.