# Would slowing Julia's release cadence improve ecosystem quality?

**URL:** <https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114>\
**Category:** Community\
**Created:** [August 22, 2023, 8:24pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114 "2023-08-22T20:24:20Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 23, 2023, 9:51am UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/21 "2023-08-23T09:51:22Z")

</div>

> [@aplavin](#):
>
> Meanwhile, “Julia updates breaking existing package code” doesn’t seem easily fixable.

It’s not just not easily fixable, it’s impossible. Anyone can write a code that uses an internal that some one says “please don’t rely on this, it’s not guaranteed to be stable”. All you need is a single package in an entire manifest to rely on something that they are explicitly told not to, and then boom you have this result. And many packages don’t even do this accidentally, many packages that integrate with the compiler do this on purpose and then directly version which versions of Julia they work with.

---

<div class="post-metadata">

**Author:** ![xiaoxi](https://avatars.discourse-cdn.com/v4/letter/x/a9adbd/32.png) [@xiaoxi](https://discourse.julialang.org/u/xiaoxi)\
**Post date:** [August 23, 2023, 10:16am UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/22 "2023-08-23T10:16:18Z")

</div>

> [@ChrisRackauckas](#):
>
> The con is that the cutting edge AD stuff would tell people to use Nightly because the feature they needed for the performance update was added to Julia 2 years ago but a release hasn’t been made.

Would it make more sense to slow down Julia releases after an LTS that includes most of the machinery needed for those cutting-edge AD libraries?

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 23, 2023, 10:32am UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/23 "2023-08-23T10:32:20Z")

</div>

> [@What steps should the Julia community take to bring Julia to the next level of popularity?](https://discourse.julialang.org/t/what-steps-should-the-julia-community-take-to-bring-julia-to-the-next-level-of-popularity/101389/418):
>
> It’s not just not easily fixable, it’s impossible.

Well, _absolute_ compatibility is indeed impossible because it means no changing anything. But in Julia, the contrast between advertisements like

> [@jacobusmmsmit](#):
>
> If it works now it will work until Julia 2.X.

and reality does feel like a betrayal when encountered for the first time.  
In reality, one basically cannot keep code intact for a few years and hope it works with new julia versions. Recent breakages have happened in packages so popular and with so many dependents, that it was kinda impossible to avoid in any nontrivial environment.

As I said multiple times already, I’m not arguing this is fundamentally wrong! It’s just that too strong promises shouldn’t be advertised as much.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [August 23, 2023, 10:42am UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/24 "2023-08-23T10:42:43Z")

</div>

> [@aplavin](#):
>
> and reality does feel like a betrayal when encountered for the first time.

I think you are exaggerating a bit the problem here. As a package developer that does not use internals, my packages install and run just fine on any version of Julia, and with any version of the packages. Some of the packages rely on LoopVectorization, but that dependency gets sorted out correctly because it (LV) has the correct lower bounds for Julia. Yes, I cannot use a manifest file that relies on a newer LV version and my package because the newer LV is only compatible with newer Julia, but no Julia update broke my packages ever.

So, while these packages that rely on internals (or any new features) keep their _Julia_ bounds correct, we other package developers should not experience these compatibility issues in everyday use, and that’s why I do not see them ever.

ps: Of course your experience may differ depending on the toolkit you rely on. But I think that the `add RandomPackage` is pretty robust in Julia, in any version. I would bet that one needs to do cherry picking to find a package that stopped working in that sense in a new Julia version, that worked previously.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 23, 2023, 10:47am UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/25 "2023-08-23T10:47:47Z")

</div>

Let’s take one concrete recent example out of many. All environments that include `StatsBase` with versions pinned a few months earlier or before, stopped working in Julia 1.9. How I, as a user, know that code in such fundamental packages isn’t supposed to stay compatible, and I need at least regular package upgrades so that my program works on newer julias? Especially given the widespread advertising of “everything continues to work for Julia 1.x, users don’t need to do anything”.

> [@lmiq](#):
>
> I would bet that one needs to do cherry picking to find a package that stopped working in that sense in a new Julia version, that worked previously.

Maybe it’s cherry picking, but these cherries are among the most popular packages (:

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [August 23, 2023, 10:49am UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/26 "2023-08-23T10:49:53Z")

</div>

> [@aplavin](#):
>
> with versions pinned

The cherry picking I think is in this phrase. If you are pinning, then yes. That I agree. If you aiming to exact reproducibility, you need to keep the Julia version (and it is possible that this should be better advertised). But I would guess that you can install the same packages (without pinning) in the newer Julia versions, for the same environment.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 23, 2023, 10:53am UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/27 "2023-08-23T10:53:49Z")

</div>

Well, “update packages and most of them _(those updated by the authors)_ will work on new Julia version” doesn’t say basically anything on Julias backwards compatibility.  
Yes, one can upgrade all packages (and effectively has to!), but this involves dealing with package updates — not only Julia updates: new bugs, deprecations, regressions, etc.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [August 23, 2023, 10:57am UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/28 "2023-08-23T10:57:40Z")

</div>

Yes, you’re right. I think I always saw that claim in the sense as not having to worry myself as a package developer that _my_ package would suddenly stop working in a new Julia version because of a breaking syntax or feature change.

---

<div class="post-metadata">

**Author:** ![fatteneder](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fatteneder/32/33991_2.png) [@fatteneder](https://discourse.julialang.org/u/fatteneder)\
**Post date:** [August 23, 2023, 11:25am UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/29 "2023-08-23T11:25:25Z")

</div>

> [@jacobusmmsmit](#):
>
> If it works now it will work until Julia 2.X.

I think a more precise phrasing should be

> If _your code only depends on public methods from `Base` and `stdlib`_ and it works now it will work until Julia 2.X.

As all past discussions about this revealed, all bets are off once you add any other dependencies.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [August 23, 2023, 2:29pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/30 "2023-08-23T14:29:19Z")

</div>

We’ve talked about all this before in a previous thread, but to be clear, lower bounds on dependencies is not the only issue with reproducibility in the Julia ecosystem. The other issue is this:

Any package that relies on internal functions from Base or stdlib should place an upper bound on the Julia version in the `compat` section of their Project.toml, but I’ve never encountered a Julia package relying on Base internals that does this.

To take a specific example, DataFrames.jl extends `Base.dotview` and `Base.dotgetproperty`, which are not part of the public API of Base Julia. But DataFrames does not place an upper bound on the Julia version. Version 1.6.1 of DataFrames works on Julia 1.9. Suppose I have a package ReallyCoolPackage.jl with the following `compat` section in my Project.toml file:

```julia
[compat]
julia = "1.6"
DataFrames = "=1.6.1"

```

Now suppose Julia 1.10 comes out and `Base.dotview` and `Base.dotgetproperty` have been removed from Base. If I upgrade to Julia 1.10 and I run `] instantiate --project` in my package directory, the dependency resolver will happily install my environment. But when I run the code in my package, it will be broken if it uses any part of the DataFrames API that internally calls `Base.dotview` or `Base.dotgetproperty`.

The point here is that an environment that is nominally feasible is actually infeasible because a dependency that uses Base internals did not upper bound the Julia version.

I’m not trying to pick on DataFrames, that’s just the example of internals usage that I’m most familiar with. Unfortunately, I think the practice of using Base internals without upper bounding the Julia version is actually pretty common in the Julia ecosystem.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [August 23, 2023, 2:48pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/31 "2023-08-23T14:48:50Z")

</div>

> [@CameronBieganek](#):
>
> lower bounds on dependencies is not the only issue with reproducibility in the Julia ecosystem. The other issue is this:

I don’t think what you are describing is something that has to do with reproducibility. For that, I would say you should really run with the same Julia version. You seem to more describe possible breakage when updating Julia versions which is in my opinion a different matter.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [August 23, 2023, 3:09pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/32 "2023-08-23T15:09:00Z")

</div>

Yeah, I guess “reproducibility” is not the right word here. The issue is “environments that are installable and should work based on the declared compat bounds, but which don’t actually work because the compat bounds are wrong.” I’m not sure what the word for that is.

Of course that sentence applies to both lower and upper bounds, whereas I was focusing on the lack of appropriate upper bounds in some packages.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [August 23, 2023, 3:12pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/33 "2023-08-23T15:12:32Z")

</div>

For what is worth, are there good solutions for these kind of problems in other languages or ecosystems?

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [August 23, 2023, 3:21pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/34 "2023-08-23T15:21:56Z")

</div>

> [@CameronBieganek](#):
>
> but I’ve never encountered a Julia package relying on Base internals that does this.

I used to do so for LocalRegistry, although technically it relied on Pkg internals rather than Base internals. The major drawback of this is that your package now won’t be installable in unreleased Julia versions (think master or nightly builds) which you haven’t yet bumped your compat bound for, because you don’t know whether it will stay compatible with those versions.

Btw I dropped the upper bound recently because I removed the dependencies on the internals, except in the tests.

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [August 23, 2023, 3:48pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/35 "2023-08-23T15:48:50Z")

</div>

Is there any way to programmatically find out whether some function is in official API or internal?

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [August 23, 2023, 4:10pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/36 "2023-08-23T16:10:32Z")

</div>

Not yet: [Taking API surface seriously: Proposing Syntax for declaring API · Issue #49973 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/49973)

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [August 23, 2023, 4:36pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/37 "2023-08-23T16:36:00Z")

</div>

If there were such a way (and doesn’t look like it comes soon), then it would be possible to automatically request an upper limit for Julia version (or for package dependencies) during the package registration.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [August 23, 2023, 4:39pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/38 "2023-08-23T16:39:54Z")

</div>

> [@Eben60](#):
>
> (and doesn’t look like it comes soon)

Well, a partial solution is work-in-progress: [https://github.com/JuliaLang/julia/pull/50105](https://github.com/JuliaLang/julia/pull/50105)

---

<div class="post-metadata">

**Author:** ![mihalybaci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mihalybaci/32/13528_2.png) [@mihalybaci](https://discourse.julialang.org/u/mihalybaci)\
**Post date:** [August 23, 2023, 6:10pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/39 "2023-08-23T18:10:21Z")

</div>

(Apologies if anyone mentioned this already, but I didn’t see it.)

My understanding, which has been stated many times in various places (IIRC), is that the “if it works now it will work until Julia 2.x” statement _is only for the Julia language itself_, not the packages. So, if I write 10000 lines of code **with no package dependencies** for Julia 1.9, then it should work in 1.7, 1.8, 1.10, etc.

As a comparison, over the course of Python 3.x, different ways to to string interpolation were added as features, including f-strings in 3.6. So if I wrote a package for Python 3.11 with the following lines

```julia
x = 1
print(f"x = {x}")

```

Then my package would break for all users still on Python 3.5 or earlier, yet no one (anyone here?) considers 3.5 → 3.6 a “breaking” Python release. So one question is, given a completely analogous situation, why is does the bar seem so much higher for Julia than other languages?

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [August 23, 2023, 6:17pm UTC](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114/40 "2023-08-23T18:17:54Z")

</div>

> [@mihalybaci](#):
>
> _is only for the Julia language itself_,

and even then, only for exported or publicly-documented behavior

I think your analogy might have the comparison backwards. shouldn’t you be looking for examples of user code that work on 3.5 and then break on 3.6+ ?

[Previous page](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114.md?page=1)

[Next page](https://discourse.julialang.org/t/would-slowing-julias-release-cadence-improve-ecosystem-quality/103114.md?page=3)