I think the key thing is for them to become aware of places where they may be depending on internals; most likely, they just updated to new internals without realizing what a problem this can cause. The better option is to surface the issue so that collectively we can try to find a way forward without depending on the internals.
In other words, I can imagine the following:
I’m writing a package, I see that something in Base would be useful, and say, “oh I’ll just call that directly”
Months go by and I forgot about this (minor? I thought so!) problem
Now someone reports that this has indeed caused them trouble
I feel guilty and decide to put more effort into distilling exactly what I need. Either I modify my package or submit an issue/PR to Julia to ensure global harmony and peace on earth.
I recall an example in which some packages seem to use internals due to some idiosyncrasies in Julia, e.g. mapreduce using pairwise reduction for some arguments and not others, „making“ people use internals from the Broadcast module even though:
It is not documented that Julia will use pairwise reduction or when, so that could technically change without being considered breaking
Broadcast is not documented, so it could stop working or be removed without being considered breaking
Even if the above breakage scenarios are unlikely, a better thing to do would be one of the following:
Write a package for pairwise reduction that covers all use cases and use that where pairwise reduction is required and expected
Upstream a feature to Julia that makes pairwise reduction the default in all cases and document this behavior, then release this in a minor version — thereby this becomes part of the public API
My guess is that the first was too much work and the second was deemed to be too much trouble for little gain, and so we are where we are now. But there is definitely something actionable here.
The associativity of the reduction is implementation-dependent. Additionally, some implementations may reuse the return value of f for elements that appear multiple times in itr. Use mapfoldl or mapfoldr instead for guaranteed left or right associativity and invocation of f for every value.
So the alternatives are anyway mapfoldr/mapfoldl, not internals.
If you’re reducing floating point numbers and accuracy matters, you really want pairwise reduction. mapfoldr and mapfoldl are not useful alternatives in many cases.
I was talking about a case where pairwise was wanted though.
This uses pairwise reduction, but it is not always called, as witnessed by the following lines and the comments.
All of these are currently implementation details, but the point I was making was that certain packages assume that pairwise reduction is used for some call signatures, and use even deeper internals when it is not used in order to get it.
Once per day, we run PkgEval (using the latest Julia master) against all Julia packages that in the General registry.
It would be great if we could get package authors more involved in that process. A failure on the nightly PkgEval can be a way to detect breakage early on.
And do what (from the package dev side)?
Sure, he can update the package code so that it works on the new julia version, but older versions of the package will remain broken on newer julias forever.
Also, he can do that now just as well, even without caring for nightlies at all — just wait until the release and update your package if it broke something.
Note that PkgEval runs on the current state of registry and packages, so it doesn’t detect when Julia update actually broke a package, but then the package got updated to fix this breakage.
If we consider packages as they were just two years ago, many wouldn’t even precompile on current Julia.
It’s important to be forward-thinking. Sure, this has been a problem in the past. What route do you see to preventing it from being a problem in the future? Are you saying it’s just hopeless and we should all give up trying?
To be concrete: suppose, for the purposes of argument, that I accept that Julia 1.9 doesn’t work on very many packages with their releases that were current in the Julia 1.0-1.1 time frame. In my mind, the question is: what do we need to do to move the ecosystem in a direction so that package releases within the next year or so continue to work on Julia releases up to 1.18 or so? Why doesn’t reporting past “transgressions” count as a way of moving things to a better place in the future?
In other words, I’m prepared to throw old package releases under the bus—of course we’re not going back in history and fixing them. I’m much more interested in ensuring that future package releases don’t suffer from the same fate.
so it doesn’t detect when Julia update actually broke a package, but then the package got updated to fix this breakage
This is why I’m beginning to think it’s actively counterproductive for packages to test against nightly. Only a few packages, like the Revise stack which is used to develop Julia, should probably be doing that. Heck, CodeTracking is in the Revise stack and I’m deliberately not fixing a breakage on nightly because I want that to be blocking for a PkgEval run.
suppose, for the purposes of argument that I accept that Julia 1.9 doesn’t work on very many packages with their releases that were current in the Julia 1.0-1.1 time frame
It would be good to clarify the actual extent of this to keep some perspective here. A lot from 1.0-1.1 still works. There must be more than a dozen packages I maintain that haven’t had code changes in multiple years and still work.
I occasionally have to fix something because of a change on nightly (or a julia dev does it for me from testing ), but from this limited experience its only happened in about 5-10% of packages in the last 5 years.
Given this will never practically be 0%, what are we aiming for?
I think the argument is that there are a handful of “deep” dependencies (LoopVectorization, Cassette) that depend on internals which means everything downstream of them gets infected by the need to update your manifest between Julia versions. I think the aim would be for all the “core” ecosystem packages (by some definition of that idea) would work out and upstream a stable public API for the stuff they do or switch to existing public methods where possible.
Would it be possible that these few internals-dependent packages start setting the upper limit of Julia version to be equal to the current minor version? Then the downstream package should set the lower limit for the package dependency to the current LoopVectorization etc. version. As soon as a new Julia version (pre-)released, these internals-dependent packages check compatibility and release a patch version, only bumping the compat entry for Julia, should there be no problems with the new version.
Interesting suggestion @Eben60. I like it enough that I’m floating a trial balloon. As I explain in that PR, I’m not certain it accomplishes the magical thing that people seem to want, “make old project/manifest versions work on new Julia versions,” but if nothing else it will directly force people to choose between sticking with an old Julia release or updating their packages—there’s no “let’s run the code and see if it still works.” I’ll be interested to see what kind of commentary that PR gets.
@Eben60 I really like the idea. I just went looking at a few packages to see how much maintenance work it would add for me
But adding friction like manually managing compat to using internals is a hidden benefit, it should be annoying to use them. I will probably remove all the internals I use instead of doing that.
Worth noting that upper-bounding the latest release of a package doesn’t really do anything except get Julia to install an older version (without that upper bound), which is even more likely to break. It’s necessary to retroactively cap the Julia version compat for all registered versions of that package (which, I think, we did for all packages when 1.0 was released).
Hmm, I’ve been suggesting for a while now that packages using Julia internals need to upper bound the Julia version in their [compat] section, both in this thread and in the previous thread. But I’m glad the concept is finally starting to get some buy-in.
), but from this limited experience its only happened in about 5-10% of packages in the last 5 years.
But I’m glad the concept is finally starting to get some buy-in.