That is a very good question. I agree with you that it is important to support multiple Git hosting providers.
This is a great idea! Unfortunately I personally do not have the bandwidth to develop this script, as my energy is focused on maintaining the automated GitHub bot. But I think a standalone script that you can run locally is a very good idea, and I think that people would definitely use it.
Hopefully someone reading this thread will see your idea and have the time and energy to implement it!
I’ve been wondering if its possible to build a similar bot for upstream package authors to understand the magnitude of any breaking changes they make. Like running the tests for all the dependent packages when you do @JuliaRegistrator saferegister, and force a manual pull request merge if you broke anything.
Not sure if this is already mentioned already, but the auto-merge condition does not disallow you to have:
[compat]
UpstreamPackage = "0"
(But I’m not sure if this is an overlook of the auto-merge condition or my misinterpretation.)
It still introduces the upper bound. But, if the upstream declares that their API is stable (in a “real sense”), you should be fine with bounding the major version.
IIUC, this “solution” works now, without any additional tools.
A downside is that lower bounding this way is impossible (at least until the version range is implemented). However, given that Pkg.update and Pkg.add is rather aggressive ATM, that’s probably not the problem. Also, since package authors effectively declaring that they don’t care about compatibility by using = "0", it’s probably OK for them to not encoding any kind of compatibility information. (If they care UX a bit more, it’s not hard to emit warning via __init__. It’s probably easy to create SoftCompat.jl to warp it in an easier interface.)
Having said that, I’m not sure if = "0" is the right direction. CompatHelper.jl seems to be a much better solution once it is stabilized and more things are automated.
Sorry if everyone is super offended that I don’t care about your reliability needs, but it is not actually my responsibility or job description.
When something breaks, I immediately want to know about it and fix it.
As such, I have decided to discontinue registering new versions of my packages.
I no longer want the burden of being responsible for your needs, unless I am paid for the service.
Therefore, you can check out the master branches if you want to get an update from now on, or pay me to make a registered release in the future.
As such, the automated testing may not work anymore, so you will have to do the testing with updated dependencies yourself, since I am not paid to maintain and register and automate the testing.
Determined users will find a way to get the code from my repository and use it, but it is not my responsibility at the moment to maintain releases. If you want to do automated testing with a new release of my packages, please pay me for the registering of it and ensuring the reliability.
You could consider to still tag releases to make life easier for your packages’ users without registering them – that should have exactly the same overhead you had before the strict-bounds-requirement was introduced to General.
I think the biggest disparity between the two views here is how many and what type of issues would you like users in the wild to discover:
Your approach says you’d like all users (novice and experienced) to discover all the issues (features as well as compatibility).
The suggested approach suggests to spare all the users from compatibility related issues.
Maybe I should just privately distribute my software to the users I care about, then random strangers with no stakes in it don’t need to worry about it at all.
It’s nice to finally have some light shed on this issue by @kristoffer.carlsson after all the arguing in the dark. The results are perhaps a little better than I was expecting, but not so good that I don’t have serious doubts about whether there is a significant fraction of the ecosystem which is inappropriately bounded, and, more importantly whether this problem will become exacerbated as more bounds are placed.
At the very least, I think we desperately need a tool that gives similar lists very explicitly whenver someone is experiencing one of their packages being held back. I also think we need an option which, at least for development purposes, lifts the compat bounds of only a specific package, preferably even if that package is not directly referenced in the Project.toml.
I think the root of all the concern here is not that strongly incentivizing upper bounds could never be a good practice, but rather that it simply isn’t a good practice at this particular moment in the Julia community. If the ecosystem were a lot more mature and lots of packages were, say in their 3.0 series, then I would say “Yes, definitely if somebody has a package that’s at 3.6 and goes to 4.0 that is very likely to mean it will break”. That just isn’t what my experiences have been with Julia packages. I’d even go as far as to say that a lot of people are using the 0.x releases to mean “don’t take the semver on these tags that seriously” (I’m not defending this practice if it even exists, and I perhaps am a bit guilty of it myself, and I think one lesson we should all take away from these conversations is that we should all take semver very seriously). The way things are, I still am very much of the mind that upper bounds will cause more harm than good. It may not always be that way, perhaps it won’t even be that way in a year, but for now it still makes me extremely nervous without better tooling.
To be fair, according to the semver spec, that’s literally what it means. Even the thing where patch releases are assumed to be non-breaking is an invention of the Julia package manager.
Basically, the entirety of the problem is that any packages which generally intend to maintain a stable API release-to-release should be released as 1.0-series packages regardless of the perceived “maturity” of the package. Most popular pre-1.0 packages in the ecosystem fall into this category. If the package ecosystem is going to follow semver, we have to stop ascribing extra meaning that isn’t there to version numbers.
I am in favor of the General registry forcing compat bounds as a means of encouraging the Julia community to adopt the norms required by semver and hurry up and release 1.0.
I doubt too many will go along with this, because it is certainly confusing, but might it make sense to require compat bounds only for packages which have releases ≥ v"1.0"?
I could see the argument for this if there was an additional requirement that such packages must have a Manifest checked in so that there’s some known good set of dependencies.
At this point I would be happy to go along with just about anything that drops the requirement of bounding pre 1.0 packages. If only to save myself the pain and suffering I know I’m going to go through at least a couple of times over the next couple of years trying to upgrade my dependencies.
@ExpandingMan I really like your comment about this and I would completely agree.
It would make sense to have slightly different standards for compat bounds for software that is 0.x
Also, note that I am willing to make my compat bounds safer at some point, but right now it just really doesn’t matter to me for the particular 0.x package I am registering… and I am not convinced that there will be an actual upcoming issue for anybody using my package. If there was an issue, I could surely have it fixed within a few hours of it coming up.
I definitely see the benefit for being safe with releases, which is why I decided to no longer take on the responsibility of making official releases.
It seems that the community is now expecting registered packages to be safe for enterprise use.
If you are an enterprise customer using free software, you should probably be talking to the developers about your plans and the reliability you are seeking first. Nobody has approached me with their enterprise reliability needs specifically, so I don’t believe it is needed for my research software.
It would be nice if there were a way to do this through Pkg.jl. Something like compat [package] that returns the current compatibility of a package and available dependencies would be nice.
I think things breaking is annoying for everyone, not just companies. In fact, enterprises are the most likely to have their own systems / rules etc that prevent breakage. I really think these compat requirements are more for everyone else…
My impression from reading this thread is not that anyone is offended that you don’t care about your users. To the exent anyone is offended, they’re offended that you seem to want Kristoffer and the other folks that maintain the general registry (also for free btw), to do all the work maintaining the reliability of your package and everyone else’s.
Great! Can you then also stop commenting on this thread then, since it no longer applies to you? You’ve made this “threat” several times now, seemingly in the hopes that someone will say “Oh no, please don’t go! We just want to keep you happy.” No one in this thread is asking you to maintain your packages in the general registry. If you want to do so, then there are standards. If you don’t want to, no worries - happily, those same people you seem to be mad at for setting these standards have designed an amazing package manager that make is super simple for others to use your code, even if it’s not registered. To reiterate:
The irony is, when people try to use your unregistered packages and run into errors, then come to Discourse and Slack asking for help, it will be folks like Kristoffer that will spend their free time trying to help them solve the issue.
Given my respect for @ExpandingMan, his concerns give me a bit of anxiety for the ecosystem as a whole. But I also haven’t seen a compelling response to this point. Given the choice between a bunch of sub-optimal options, making the default be the one where using ] add SomePackage works, even if you’re prevented from getting the latest version, seems preferable to the one where it’s broken for unknown reasons. And I’m confident that while we wait for good tooling for the rest of us, most of the commonly used packages are actively maintained enough by skillful enough people that their compats will be kept up to date.
I doubt anyone is actually offended, and your statement also does not represent my expectations, my personal expectation was initially that the general registry wasn’t a very seriously vetted place, where having a package registered in the registry didn’t imply it had some level of reliability.
Now, there is a certain level of reliability expected, and this is in fact a change in the status quo.
That’s certainly something worth commenting about with a few bytes of text.
I think the point you are referring to is that @fredrikekre seems to have been suggesting that the inability to update packages is easier to resolve than errors due to incompatible new versions of dependencies. That simply hasn’t been my own experience. Usually I have some idea of where an error is coming from thanks to stack traces, and the package authors have an even better idea when provided with said stack trace. In my experience it is also usually easier to deal with a hack to circumvent some specific issue rather than decending into the abyss of resolving a dependency nightmare. I’m sure much of this is due to the fact that we have better tools for tracing down actual errors than dealing with dependencies, but that’s part of my point, requiring these bounds seems extremely premature. I feel that, for it to work as expected, we need immaculate tooling, and encouraging everything to be bounding with the limited tooling we have for this now seems, to me, to be inviting trouble.
I should say that, in case anyone has inflated ideas of my importance in the community, I am very far from one of the most active package developers. My experience is limited. Sadly, most of what I’ve produced is code for my job that will never see the light of day. That said, one of the most frustrating issues I have found with the Julia ecosystem is needed something updated which I can’t get updated. Today this does not happen often, but it is often that I want a new version of a package. I find that, in the current state of the Julia ecosystem, the vast majority of the time you are far, far better off with newer versions than with older ones, even if you have to resolve a few minor issues to get there. I have no doubt this reflects the youth of Julia’s ecosystem, that most other active language ecosystems are not like this and that even for Julia this will not always be true. For the time being, it seems to me that often the best advice for fixing something is to first update everything, because there’s a good chance that whatever you’re experiencing is something that package authors have been working on.
My fear all along is that the architects of this, while obviously very well intentioned, are simply taking semver a lot more seriously than much of the Julia community has to date, and perhaps have unrealistic expectations about how often package authors will be willing to make patches purely for the sake of updating the bounds of their dependencies. Those on the other side of this argument continue to say things that I completely agree with in principle, but which distress me because they don’t jive with my own experiences writing Julia code (in which I usually feel much better off having the latest versions of things).
That’s another good point. The General registry is certainly not thoroughly vetted. I don’t see anything wrong with that, if anyone wants a well-vetted registry, they can create one. However, when one considers that package bounds alone are only a small part of ensureing reliability, and that things can break for a host of other reasons, is it really worth the potential heartache of trying to enforce these bounds in General? Wouldn’t this be more appropriate in a hypothetical future registry that actually is vetted for reliability?
