# Did Julia community do something to improve its correctness?

**URL:** <https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515>\
**Category:** General Usage\
**Created:** [August 5, 2023, 7:16am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515 "2023-08-05T07:16:42Z")\
**Posts on this page:** 20\
**Page:** 11

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [October 27, 2023, 9:02pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/246 "2023-10-27T21:02:18Z")

</div>

Feels like the discussion is running in circles. One of the things that bothers me a bit, is that discussing away bugs seems a bit simpler in Julia.  
First, its interfaces are usually not formally defined or enforced and accordingly one can always argue whether its a bug or not. E.g., what would you expect when calling `cycle` on a stateful iterator: a) It caches all values and reuses those (which could lead to unexpected memory leaks – just search for _lazy IO_ in Haskell) or b) it throws an error (making it well defined, but essentially useless) or c) it does something which you might or might not expect (putting it in the ream of _undefined behaviour_ which is morally defensible as no correct behaviour seems possible anyways)?  
Secondly, the assumptions behind generic interfaces are also not well specified and some implementations make slightly different choices than others. I would certainly agree that the implementation

```julia
reverse(z::Zip) = Zip(Base.map(reverse, z.is)) # n.b. we assume all iterators are the same length

```

is wrong as its assumption is squarely at odds with the docstring of `zip`

```julia
  zip(iters...)

  Run multiple iterators at the same time, until any of them is exhausted.

```

Generic programming enables very composable code, but also requires higher-levels of abstractions to work flawlessly, and at points, some Julia code appears a bit careless in what can be assumed and what not. While I don’t expect Haskell like abstractions and laws, some documentation around the assumed invariances in generic interfaces could be helpful at times.

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [October 27, 2023, 9:09pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/247 "2023-10-27T21:09:46Z")

</div>

Something like [GitHub - Keno/InterfaceSpecs.jl: Playground for formal specifications of interfaces in Julia](https://github.com/Keno/InterfaceSpecs.jl) would be a great start.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 27, 2023, 9:37pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/248 "2023-10-27T21:37:57Z")

</div>

It wouldn’t surprise me to find that Julia lets people hit edge cases a lot faster than other languages do.

I’m writing a mathematical modeling book, in it for one example I fit a posterior distribution over Chebyshev Polynomials using ApproxFun.jl and Turing.jl. Because of the science we expect these Chebyshev Polynomials to be non-decreasing I do a trick to make this work.

Thanks to the magic of ApproxFun I can just make a function f, then calculate g = `Integral()*(f^2)` to get an integral of f^2 as an ApproxFun. Could I do this in R? Not a chance. Could I use R and call out to Stan? Well, I asked the Stan people to add Chebyshev functions in 2013 or something, and they never did and just closed the bug after a while. Could I do this in Python? I don’t know but if I did it’d probably not be in Python per se, it’d be in numpy or some big library of C/C++ functions callable from Python.

It’s magically perfect in Julia.

So, _for me_ Julia enables me to do things that make sense and are sophisticated stuff that’s virtually impossible in other languages in the same niche. But when the realm of things I might try is _combinatorially bigger_ it certainly makes for a bigger potential bug space as well. I should pause probably before trying to take forward diffs of Matrix Algebra of Unitful units for example.

That being said, while I don’t program in it all day every day, I have _never_ encountered a serious correctness bug that secretly gave me wrong answers that I’m aware of. I’ve definitely found some bugs that crash weirdly or similar, but never just returning a secretly wrong answer. I think a couple times I was able to do something like make a Loess.jl fit return NaN.

When I used to program in C/C++ decades ago, you’d constantly be running your code and debugging use-after-free or NULL pointers or type casts or memory leaks or whatever. In the last few decades doing data analysis and such in R I’d just never do anything actually requiring computing in R itself, everything is call out to some C/C++ code.

I suppose perhaps if there were an actual competitor to Julia and it were substantially less likely to give me wrong answers, I’d be upset about how Julia secretly returns wrong results. That isn’t the scenario I find myself in though.

I still think a repository that stress tests large swaths of the Julia ecosystem would be a good project though. I’d like to be able to say “in these 800 heavy tests of Julia doing statistical calculations and simulations, there’s exactly 3 bugs that have been found and 2 of them cause a crash while the third causes a minor bias in some monte carlo output that can be fixed by workaround X”

or stuff like that.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [October 27, 2023, 9:58pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/249 "2023-10-27T21:58:23Z")

</div>

There is a difference between the incorrectness of the _language_ and the incorrectness of the _compiler_. A lot of people here conflate those two.

Edit: Btw, what @Sukera had in mind is yet another category, incorrectness of the program. I think that Julia does not have an incorrectness problem as a language. The compiler has bugs, as every other compiler does. The general packages (and the standard libraries, in some cases) may have program correctness bugs. But show me a C++ library without bugs…

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 27, 2023, 11:17pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/250 "2023-10-27T23:17:29Z")

</div>

I don’t know. In Julia the compiler and the language are sort of integrated. In Fortran I had the opposite experience. You can even get Fortran comitee people telling you that what you wrote is not a Fortran program, even if the syntax is supported by a major compiler for decades.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [October 27, 2023, 11:22pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/251 "2023-10-27T23:22:58Z")

</div>

Are there bugs in the Base language where code silently returns results that are definitely and objectively incorrect? Yes, sure, I even have one issue filed quite some time ago – [skipmissing on Dict: inconsistent results · Issue #48379 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/48379) (adding to the small heap of yet-unfixed issues in this thread (: ).  
But this is the only one in Base I encountered over a few years of very active Julia usage, such bugs definitely don’t affect my general perception of Julia.

Popular libraries can also have silent bugs of course, they aren’t common but can happen – and unfortunately sometimes can take embarrassingly long time to fix (eg [https://github.com/JuliaStats/StatsBase.jl/issues/518](https://github.com/JuliaStats/StatsBase.jl/issues/518)). Again, this is the only silent bug in Julia libraries I encountered (looking at the history of github issues).

Of course, making software bug-free would be great, but that doesn’t seem possible for now in general. I wonder what are concrete suggestions that could help reducing the amount of bugs in Julia, to gain another advantage wrt other languages + libraries.

---

<div class="post-metadata">

**Author:** ![ParadaCarleton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paradacarleton/32/20005_2.png) [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)\
**Post date:** [October 28, 2023, 2:49am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/252 "2023-10-28T02:49:10Z")

</div>

> [@jakobnissen](#):
>
> I’m also at the point where I do recommend Julia to colleagues with a caveat: I tell them that it still has rough edges, and feels like a language in beta.

I think we again need to distinguish between the ecosystem and the language. The language has some correctness bugs, but I don’t think they indicate a fundamental problem with Julia–it’s a . Julia base is probably a _bit_ buggier than Python or C++; I’ve found a few bugs in it. But I don’t think that’s all that unusual for a language with a community of roughly this size, and they do seem to be getting better over time.

What I’m much more concerned about is the Julia _ecosystem_, where I regularly see correctness bugs in _common_ cases, not just unusual compositions of edge cases. As another example, today we discovered that the cumulant function in StatsBase [is often just flat-out wrong for higher-order moments](https://github.com/JuliaStats/StatsBase.jl/issues/899), and cumulants aren’t exactly a niche feature.

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [October 28, 2023, 8:36am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/253 "2023-10-28T08:36:57Z")

</div>

I see you happily discussing onwards in cycles, how about taking a step back and figuring out what you want from this discussion?

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [October 28, 2023, 8:41am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/254 "2023-10-28T08:41:03Z")

</div>

Seems (from your source) that Python (SciPy) has exactly the same « bug », but « hides » it by returning only 4 cumulants? So what about « correctness » of Python (ecosystem)?

Is this here really a « language/ecosystem » « correctness » issue, needing a so looong discussion?

Note : I do not imply SciPy is incorrect, I do not know the rationale to this 4 limitation (would be interesting to know of related discussion if it existed). What I mean is it is not positive for the language to search examples that show that every language has same troubles

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [October 28, 2023, 8:58am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/255 "2023-10-28T08:58:20Z")

</div>

_Quoting just because it triggered my thought. This is not against the quoted person._

> [@](#):
>
> such bugs definitely don’t affect my general perception of Julia

But discussion like these do affect the perception of Julia on a wider audience, and not to a good one.

OP started with a vague claim implying everything and nothing even putting responsibility on the community. This is a pattern happening now and then in the past few years. Sometimes OPs are happily steering the discussion into the negative or like this one, the community does it on its own. **Nothing from OP since discussion started!**

It’s even unclear what this is all about from an outside view. Of course every single person here has a clear view of their issues but the overall conciseness is quit low.

These discussion tend to be non productive. I can’t see how this brings any benefit and anything we can work on. At the end it’s another _proof_ that Julia is not mature.

We should be aware of this possible outcome of discussions like that.

---

<div class="post-metadata">

**Author:** ![ParadaCarleton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paradacarleton/32/20005_2.png) [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)\
**Post date:** [October 28, 2023, 6:32pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/257 "2023-10-28T18:32:30Z")

</div>

> [@jdad](#):
>
> Seems (from your source) that Python (SciPy) has exactly the same « bug », but « hides » it by returning only 4 cumulants? So what about « correctness » of Python (ecosystem)?

That’s a completeness issue, not a correctness issue, which is both different and much milder. Saying “Sorry, that feature’s not implemented” is a whole different kind of problem from _giving the wrong answer_. I really don’t think we want to go down the route of calling every missing feature a “correctness bug,” given how many features haven’t been implemented yet in Julia (compared to Python/R).

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [October 28, 2023, 7:07pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/258 "2023-10-28T19:07:39Z")

</div>

> [@oheil](#):
>
> It’s even unclear what this is all about from an outside view

> [@oheil](#):
>
> I can’t see how this brings any benefit and anything we can work on

> [@mschauer](#):
>
> figuring out what you want from this discussion?

personally, what I would prefer to influence is community priorities, in that

1. when there is a tension between fixing an edge case/ bug and performance, generally the bugfix is preferred
2. when there is a tension between fixing an edge case / bug and semver, generally the bugfix is preferred
3. when there is a tension between fixing an edge case / bug and genericness of implementation, generally the bugfix is preferred

I have seen it happen quite a few times that some pr pops in the tracker that fixes a bug (or something that is at least _arguably_ a bug) and either stalls or is outright rejected because it leads to a 10% performance regression in such and such use case, or is pedantically minorly breaking semver, or would require runtime checks inside the implementation rather than relying only on method signature

---

<div class="post-metadata">

**Author:** ![Tarny\_GG\_Channie](https://avatars.discourse-cdn.com/v4/letter/t/3bc359/32.png) [@Tarny\_GG\_Channie](https://discourse.julialang.org/u/Tarny_GG_Channie)\
**Post date:** [November 2, 2023, 12:40pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/259 "2023-11-02T12:40:44Z")

</div>

Yes. On the other hand, it is not good to simply call the library perfect because it does nothing and therefore does not have bugs. The point was that Scipy also had the same issue, so it just cut it by limiting its own features.

---

<div class="post-metadata">

**Author:** ![ParadaCarleton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paradacarleton/32/20005_2.png) [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)\
**Post date:** [November 2, 2023, 5:58pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/260 "2023-11-02T17:58:30Z")

</div>

> [@Tarny\_GG\_Channie](#):
>
> Yes. On the other hand, it is not good to simply call the library perfect

I didn’t call the library perfect, I just said that this particular function isn’t returning incorrect answers (unlike the Julia equivalent). There’s a clear ordering here: wrong answer \< error \< correct answer.

---

<div class="post-metadata">

**Author:** ![ParadaCarleton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paradacarleton/32/20005_2.png) [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)\
**Post date:** [November 2, 2023, 9:10pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/261 "2023-11-02T21:10:44Z")

</div>

> [@adienes](#):
>
> I have seen it happen quite a few times that some pr pops in the tracker that fixes a bug (or something that is at least _arguably_ a bug) and either stalls or is outright rejected because it leads to a 10% performance regression in such and such use case, or is pedantically minorly breaking semver, or would require runtime checks inside the implementation rather than relying only on method signature

IIRC Steve McConnell calls this “cargo cult software engineering”–things like:

1. Thoroughly reviewing or nitpicking new PRs rather than addressing bigger issues in existing code (forgetting the purpose of code review is to fix bugs)
2. Obsession with keeping packages “lightweight” by refusing to take on even widely-used, quick-to-compile dependencies (forgetting the purpose of lightweight packages is to reduce compile time and fragility)
3. Religious adherence to written contracts of compatibility/API (forgetting the purpose of avoiding breaking changes is to reduce maintenance burden)

Even much more well-established languages are more willing to make big changes than Julia is, despite having much larger codebases to maintain. For example:

- Python breaks some code every minor update; they just give 3 years’ warning, more than enough time for even poorly maintained packages to fix issues.
- Dart literally switched from dynamic to static typing and abolished null pointers (two massive changes) in the 1-\>2 shift. A few months ago, they removed `f(name: arg)` for named keyword arguments.

Meanwhile, I think [this](https://discourse.julialang.org/t/psa-julia-is-not-at-that-stage-of-development-anymore/44872) is the most common reply to almost any request to fix a problem. And changes that don’t break the Official Contract provided by docs (most notably, offset indices) are treated as being non-breaking, without any mitigating measures (like scripts or tools to help people convert, `@inbounds` deprecation, or required ordinal indexing).

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [November 3, 2023, 9:52am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/262 "2023-11-03T09:52:35Z")

</div>

> [@ParadaCarleton](#):
>
> Meanwhile, I think [this](https://discourse.julialang.org/t/psa-julia-is-not-at-that-stage-of-development-anymore/44872) is the most common reply to almost any request to fix a problem.

In my experience, the PSA is often a response to mildly breaking feature requests, not to correctness bugs that may be fixed in a non-breaking manner.

---

<div class="post-metadata">

**Author:** ![ParadaCarleton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paradacarleton/32/20005_2.png) [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)\
**Post date:** [November 3, 2023, 8:26pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/263 "2023-11-03T20:26:28Z")

</div>

> [@jishnub](#):
>
> In my experience, the PSA is often a response to mildly breaking feature requests, not to correctness bugs that may be fixed in a non-breaking manner.

Yes, that’s my point (and @adienes’ as well). In many cases, fixing a correctness bug is arguably mildly breaking for a [handful of people](https://xkcd.com/1172/) who rely on incorrect or dangerous behavior.

(In other cases, any changes would be much more breaking and probably require a 2.0 release, e.g. a satisfactory solution to out-of-bounds array indexing, but the devs have ruled out a 2.0 release for the foreseeable future.)

---

<div class="post-metadata">

**Author:** ![jd-foster](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jd-foster/32/35824_2.png) [@jd-foster](https://discourse.julialang.org/u/jd-foster)\
**Post date:** [November 4, 2023, 12:55am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/264 "2023-11-04T00:55:02Z")

</div>

Isn’t there some rule of internet discussions that says referencing XKCD means the end of the thread? I think maybe Munroe’s law?

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [November 4, 2023, 4:18pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/265 "2023-11-04T16:18:27Z")

</div>

I’d say, this particular horse has been beaten to death some dozens of posts ago. Let’s close it.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [November 4, 2023, 4:26pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/266 "2023-11-04T16:26:09Z")

</div>

Closed on November 4, 2023, 4:26pm UTC



[Previous page](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515.md?page=10)