# Did Julia community do something to improve its correctness?

Yeah. This one is kind of horrifying.

Matlab (sort of) do have it, it is found here:
https://www.mathworks.com/support/bugreports

1 Like

Thanks for adding it to the 1.10 milestoneâŚ

1 Like

Thanks for linking all of those, its good to know exactly what people mean in these discussions.

Yes probably there are problems with Julia Base from being both comprehensive and internally composable more than is often possible. Your problem of the reversed filter iteration is again that we can compose too many things together. It would be interesting to know if you can do anything like that at all in other languages.

With your issues with `sum!` etc over aliasing arrays, I think I assumed that or anything like it would break somehow or be undefined. But I can see that this would seem pretty weird if you expected otherwise:

``````julia> x = [1, 2, 3]
3-element Vector{Int64}:
1
2
3

julia> sum!(x, x)
3-element Vector{Int64}:
0
0
0

julia> y = [1, 2, 3]
3-element Vector{Int64}:
1
2
3

julia> sum!(x, y)
3-element Vector{Int64}:
1
2
3
``````

Given I assumed that wouldnât work, I wouldnât personally class it as a huge correctness bug - so Iâm not sure how clear the definitions we are using are.

3 Likes

âI donât think it needs to be release blocking, as the bug has been open for years without too many people noticing.â

I appreciate that this specific example adienes linked to is being fixed, but itâs comments like these that do not make me optimistic of a cultural shift.

9 Likes

well, no, and thatâs the point. I think `reverse(::Filter)` should be entirely removed and `reverse(::Zip)` should error unless all of its constituent iterators are known to have the same finite length

6 Likes

Totaly. As I was getting at many of these bugs exist because you cant do these things in other languages - and that makes comparisons with other languages difficult.

Its also a problem to fix because removing it is a breaking change, and some people may productively use `reverse(::Filter)` with pure iterators!

As you mentioned we can hope that analysis can catch these at some point, but its hard. Maybe a docs warning is the best we have in the short term.

Is there any way to make Julia âbountyâ issues ( ~related to correctness ) more visible to the community?

• Where can I contribute to support a particular JuliaLang issue bounty?
• adding my 10 - 100 - âŚ \$
• Where can I see the existing bounties?
• Is it possible to integrate with Github issues? For example, can we create a âBountyâ label ( in the JuliaLang/julia repo ) to highlight its importance?

For reference, âbountyâ based development is emphasized in the tinygrad repository - as a âbounty-lockedâ label:

3 Likes

I mean, that seems like a totally unholdable claim. Nothing about âlazy filtering objectâ is unique to Julia, so why should it not be possible or be difficult to do in another language? To be quite honest, this just sounds like dodging the issue of âwe didnât think about that failure caseâ (which I take to be the core issue Yuri was originally getting at - undocumented/unexpected but quite obvious failure & edge cases).

The Julia semver stability guarantee has pretty much always been âif itâs in the manual, itâs APIâ. Iâd be VERY surprised if this thing actually existing was guaranteed. That doesnât change the fact that people may expect it to exist, but thatâs precisely the issue - what is actually part of semver and supposed to work is not at all well defined in Julia today, leading to the sorts of correctness issues/unexpected bugs that people complain about all the time (rightly so!).

No amount of static analysis can compensate for a lack of documentation of core functionality, or contradicting documentation about semantics in type hierarchies.

and these are just some of the ones I personally filed. Iâm not in the habit of keeping a list of these kinds of issues, but I do try to file them when I stumble across them.

14 Likes

Yup, and thatâs easy to audit:

https://github.com/search?q=org%3ASciML+sum!&type=code

https://github.com/search?q=org%3ASciML+prod!&type=code

https://github.com/search?q=org%3ASciML+map!&type=code

All of our `sum!`, `prod!` (not even used), and `map!` look fine. And the uses of `reverse!` are on Array or Tuple with no Filter, so that seems fine.

https://github.com/search?q=org%3ASciML+reverse&type=code&p=1

Of course it should be improved anyways, but this is why I object to the phrase of âpackagesâ. This shows about >100 of some of the most used packages in Julia do not touch the aforementioned issues. `prod!` doesnât even show up in the source code. One should qualify what exactly is the issue in every instance, because many areas of Julia are doing fundamentally different things.

But not only that, qualifying what exactly is the issue is a constructive way to help get it fixed.

This is because those languages donât include their major math libraries. numpy and PETSc have all sort of bugs and you hear about them during conference talks all of the time. The large the support surface the longer it takes to stabilize. Python is in a particularly weird position where its standard libraries tend to be so gimped that people tend to use many standard library replacements, which tend to be less supported and have quite a few bugs, and so Iâm not sure how thatâs any better.

Notably, the built in password and login libraries in Python are known to have security issues so they arenât recommended and itâs been this way since 2008 or something like that. Youâre taught as a beginning user to just avoid those modules, so therefore itâs safe? So people will define Python as safe by simply moving the boundary to not include those modules (which may end of life in 2028). Julia just has a different looking standard library since itâs historically been focused towards different developers, but in just the same way as you go further into the weeds youâll find things less supported.

The larger the surface, the more there is to do.

22 Likes

Maybe I just got used to such quirks in dynamic languages, as all of them seem to have issues where silently wrong results can be returned. Here one example from R and Python each, that had bitten me in the past:

• R: Recycling of vectors with non-matching length
• Python: Before numpy made the switch to `@`, it had `W * x` mean matrix multiplication when `W` is a `Matrix`, but element-wise multiplication when `W` is an `Array`.

Just for fun, I checked the reverse filter example using `std::ranges` in C++20. Using the stateful predicate it compiles fine and then segfaults.There are very few compilers out there that warn about impure functions â mainly Haskell and some experimental effect systems. Basically, I only trust the Haskell compiler to a good extend (have not tried Rust though) â C++ is a lost case (compiled or not).

12 Likes

thatâs interesting, thanks for trying it! still, I would prefer a (loud) segfault over getting the wrong answer

2 Likes

Me too, but unfortunately we donât always get it. At least in my experience, Julia has not been worse than other dynamic languages in that regard.
Iâm not saying these issues should not be fixed though âŚ the less quirks and corner cases the better. Not just for that reason, I like the principled approach of Haskell stating laws that have to hold for its generic types, i.e., type classes. Guess that also Julia could benefit if required properties of its implicit interfaces could be easily tested, e.g., using PropCheck or something similar.

1 Like

Thatâs an API quirk, not a correctness bug â R is designed to do that intentionally. The Julia examples like `reverse(zip(` are incorrect implementations of documented behavior. Having a compiler check the documented assumptions is a nice feature but itâs not very useful if the implementation assumptions arenât actually part of the documented interface in the first place.

3 Likes

The problem with the topic of Julia and correctness was how it lumped many issues with varying degrees of relevance into One Thing newcomers have to consider. Itâs not. Newcomers care about whether they can write practical code that gives the correct results; yes they can, which is why we get more of the subsequent questions of âhow do I optimize thisâ. Newcomers care about whether they can pick up packages that work properly; yes they can, and like any other language it depends on how actively maintained the package is.

Itâs not particularly helpful for newcomers to be bombarded by bugs in corner cases or composition with little-used packages. For example, while I do agree that aliasing should be handled instead of silent, we know not to alias with reducing operations in practice, and we would never reduce over none of the dimensions, specifically `sum!(copy(a), a)` gives the same result as `sum(a;dims=())` or `copy(a)`. This is irrelevant to a newcomer who wants to Do Something. I get that people can be frustrated by problems in areas with little attention or demand, but thatâs an issue for any language and it shouldnât be a reason to avoid a language that is proven useful.

19 Likes

Technically correct, but it has a similar effect to program rather defensively when using it.

`reverse(zip(` I would also expect to work. `filter` with an impure predicate probably not, but it does not seem to be documented what can be assumed in that case.

If you look at the example in the issue, the mutable functor in question is user-defined:

And passed into `Iterators.filter` and `Iterators.reverse`.

As Iâve mentioned above statements like âseems like a totally unholdable claimâ donât help this conversationâŚ we do actually want to know in what language you can both

1. do that easily
and
2. it correctly handles the distinction between a pure function and an impure function

Then we have a clear comparison to make, and maybe something to learn from

I thought that the thing with composability was that `reverse(filter(...))` would work without there being any explicit such method.

So either this particular example is not a composability issue, or you canât really remove this, except by writing a method saying itâs not implemented (oh, and do those cause the most deliciously discombobulating error messages).

45 posts were split to a new topic: On the behavior of reversed iterators

This is becoming a post on `reverse`, perhaps this post should be split?

5 Likes