You are assuming that a breaking change in some package’s release will affect every package equally, which is not the case at all. This will increase the strain on resolving compatible package versions.
No I’m not. I am just saying that you should at least test if it does break your package, which you can only do after the release is made.
Case in point; here is an example: Allow JSON v0.21 and bump version to 5.1.3. by fredrikekre · Pull Request #149 · JuliaWeb/GitHub.jl · GitHub . JSON@0.21 did not break GitHub. So when that (breaking according to semver) release of JSON was made I could test that it did in fact not break GitHub, then bump the version, and make a new release. Problem solved.
3 Likes
That’s a good point, I can see why you would want that.
It’s not in line with my KISS (keep it simple stupid) philosophy, but maybe there is some merit to it.
However, as I said, this will make version compatibility overall much more complicated and also increase the time required to compute version compatibility in general.
While you may be more “safe” from breaking changes with the proposed process, you are overall decreasing compatibility between all packages with this process. You are completely safe from anything breaking when a new release is made… but you also lose something by doing that.
I would rather wait until the breaking release was made, then test and then add bounds, but I can see why you want it this way. I’m not sure yet whether I am going to do it this way or not.
Frankly, I am not paid enough ($0) to really care about how my package is released or whether you merge my releases at all or whether it breaks something for you.
1 Like
While I understand the motivation of encouraging upper version bounds, it seems to expect maintainers to follow the dependencies of their packages fairly closely if they do not want to discourage their users from staying on the cutting edge (of the dependency). That makes it more costly to maintain “utility” packages that are not central to the use of many users, but helpful for some.
I would prefer to get an issue filed every once in a while that says “update to X breaks your package” rather than having to monitor dependencies for breaking changes (partly because of limited attention, and partly because I think an issue might be more likely to encourage help by others).
5 Likes
I assume tooling will help a lot here. @dilumaluthge already created CompatHelper.jl which will watch out for version updates of your package’s dependencies and will create PRs to bump the version in compat. It won’t tell you “this update breaks your package” I guess but it’s a step in the right direction.
12 Likes
That’s a very useful pointer, thank you. Actually, Travis on the PR will tell you whether the update has broken your package, no? (at least according to tests) So that should solve my problem, no?
Actually, you might be right. That would be even better. I haven’t had the chance to test the bot yet.
I find the language of “we should assume something broke” unhelpful when discussing Semver, this is an explanation that makes sense to me.
When dependency of your package releases a new major version (or minor pre-v1) , it is as if the author of the dependency has texted you to say dude, I changed my APIs, your usage might break. You have to choose how to respond. Path 1 with no upper bounds is equivalent to saying whatever dude, I'll hear about it if it ACTUALLY breaks something I (and therefore my users) use. Path 2 with upper bounds is equivalent to saying thanks dude, I'm going to stick with the old version until I check that nothing broke.
4 Likes
So consider this; we have package A (versions A@1.0 and A@1.1) that depends on B (version B@1.0). Now, the author of B has made some breaking changes and decide to release B@2.0. Lets compare “your” approach with “mine”.
With “your” approach, A@1.0 and A@1.1 has no bounds on B at all (because that is “reckless and careless”, and “incorrect”).
Lets first consider the case where B@2.0 does not break our versions of A: Phew, we guessed correctly, thats a relief, our users are safe to pkg> up and get B@2.0 without that breaking anything. Lets hope we are lucky with B@3.0 too.
Now, consider the case where B@2.0 does break our versions of A: Panic mode; quickly add bounds and release A@1.1.1 with correct upper bounds on B. Now our users are safe to pkg> update again. Right?
Well, no, because versions A@1.0 and A@1.1 still claims to support B@2.0, and users might end up with A@1.1.0 and B@2.0: Panic again, what to do? Ah! We can modify the registry and add the bounds we should have had there in the first place! All good!
Again, no, because there is a user that has A@1.1.0 and B@2.0 and everything works perfectly fine; that user happened to not be affected by the breakage. Now, by modifying the registry that user has ended up in an “impossible” state (according to the registry), and that will wreak havoc as soon as that user tries to use the package manager.
With “my” approach, A@1.0 and A@1.1 has bounds on B: since only B@1.0 exist, we require B = "1", since that is what we can verify.
Lets first consider the case where B@2.0 does not break our versions of A: Ah, nice, we can relax the bounds to B = "1,2", and make a new release A@1.1.1. And just to be nice, lets also release A@1.0.1 with the relaxed bound if someone still uses A@1.0.
Now, consider the case where B@2.0 does break our versions of A: Makes sense, B@2.0 is an (according to SemVer) breaking release, and we already have reasonable bounds so nothing to do. Let’s just put on our TODO list to fix A to be compatible with B@2.0.
Why do you prefer your approach here? The only “good” thing I can see with your approach is that it makes it a bit more simple in the (presumably) rare case that we guessed correctly about how breaking the next breaking release of B would be.
Welcome to the club 
19 Likes
@ggggggggg you’re right, that’s what it’s like basically
basically, I only tag releases as a courtesy so that other people with less time on their hands can keep up with my work in computer algebra…
if the Julia community does not want to accept my tag, then that’s up to you I guess
If the Julia community expects some kind of enterprise support for my package releases, please consider paying me for that service.
This whole discussion reminds me that I need to cajole some folks to add their upper bounds, because their packages are broken right now without them. Remarkably, some are packages with lower bounds, but not upper bounds in their compat entries.
The point is, it’s worse to deal with a broken package then it is to deal with a package that isn’t on the bleeding edge.
4 Likes
That may be true, but I often find myself stuck with a broken package because compat bounds prevent me from updating to get a bugfix. Working around it is actually quite tricky, deving packages does not solve the problem. I’ve had to clone a package manually, change the offending compat entries and then dev the local clone.
5 Likes
I think a case that I would find annoying is if I am the author of package A1, and some other person is the author of package A2. We both depend on package B. Now, the author of package B decides to make a “breaking” release which broke both of A1 and A2.
The author of A2 realized this and decided to only support the latest release of B in another release of A2. Then he/she made another release with an unrelated bug fix or new feature keeping the same B compat. On the other hand, I didn’t have enough time to look into the breaking changes in B so I decided to stick to the old version of B, which was done automatically by the upper bound I set.
Now, at this point the latest release of A1 and the latest release of A2 cannot co-exist. So what if a user wants to use both latest releases to make use of the bug fix or new feature in A2? He/she would have to nag the authors of A1 and A2 to support at least one common release of B. Now imagine the release of B didn’t actually break anything in my package A1 but I just didn’t have the time to check. Then this whole situation could have been avoided. And imagine if instead of only A1 and A2, there were many more packages. Figuring out what’s breaking what will require an uncomfortable amount of digging by one of the authors of these packages or the user into all the other packages to figure out a common release of B that they can all support or a way to support multiple versions by all these packages.
This kind of situation can cause long and sometimes unnecessary breakage time if package B did not really break A1, A3, A4, etc. I think a big part of this is psychological, where fixing things because they are broken is seen as time well spent whereas putting in time just to make sure things are still working is seen as a waste of time by some, especially if it could be avoided in those near-miss cases.
Following SemVer as suggested above is not very demanding — you don’t have to do anything to your code, just signal what happened with version numbers.
Various people have been working hard to make the tooling around these things convenient for the whole community, and they continue to be improved.
Of course you have no contractual obligation to follow these conventions — it’s just a small gesture to your users. However, if you ignore them, don’t be surprised if people consider your packages broken (of course this may or may not be important to you, I don’t know).
7 Likes
In practice, people just submit a small trivial PR to bump version bounds; when the package has a reasonable test suite then this can be considered sufficient verification and you (the author) can just press the merge button and then tag a release. Eg many people did this to various dependencies of StaticArrays the other day as they wanted to use the shiny new 0.12.
4 Likes
You are assuming that one of the users of my package A1 has enough experience to figure out that the compat of B in A1 needs to be bumped up, and then made a PR. I think this assumption is only true for a relatively small subset of core packages and other lucky packages that are used by Julia programmers who know what they are doing. Even figuring this out might be a difficult task for good Julia programmers who are not familiar with the code base of A1, and then there is A3, A4, etc. also to check.
Certainly, the scenario above assumes that users can investigate these issues and make trivial PRs (with as much community help as needed — and the norm in the Julia community is to provide a lot of help for these issues, with fast turnaround times).
I think this is reasonable to expect. I consider the skills required for this essential to use most free & open source software, and especially in Julia.
Of course, if someone really doesn’t want to invest in this, they can wait for or pay someone else. Keep in mind that in the meantime, the existing versions will just continue to work as before, so the scenario is not catastrophic.
1 Like
If the bug fix or feature in A2 is critical for the user, and if the user doesn’t have enough skill, time or money to either fix the issues in A1, A3, A4, etc. themselves or pay someone else to fix them, then this problem is indeed catastrophic for that user.
We would essentially be telling users to not use packages together that they can’t fix when versions conflict. And if they do use such packages together, then they should be ready for version conflicts, because that’s what they signed up for, even when such conflicts are totally avoidable in near-miss situations.
My own problems and issues have been personally solved and I can take care of my own compat issues. Actually, I don’t need to release open source at all, I could just work in private and not share code.
If other people consider my packages important enough to adhere to some enterprise guidelines, then they ought to pay me for my service.
If you encounter a specific issue, you are welcome to open a PR or issue about it though. Whether you consider it broken is not a priority for me, unless there is some incentive to support your needs specifically.
As I said, i anticipate that more problems could be created by having the extra guessed bounds. These problems could be avoided by properly managing bounds for actual encountered compat issues and problems.
Sorry, I find the “this is a catastrophe for me, but I am unwilling to invest a few hours into learning the skills (with a lot of community help and handholding if necessary), then 5 minutes to apply them from then on” argument unconvincing. Please recall that we are talking about people who program in a rather advanced language, they are not exactly unskilled and helpless.
I think you misunderstand: we are just telling them that they will be stuck with the current versions until the bounds are adjusted.
As explained by @fredrikekre above, the alternative is an unholy mess that would indeed break stuff for users in ways that require much more expertise to recover from.
5 Likes