Did Julia community do something to improve its correctness?

General Usage
21

Yeah. This one is kind of horrifying.

22

Matlab (sort of) do have it, it is found here:
https://www.mathworks.com/support/bugreports

1 Like
23

Thanks for adding it to the 1.10 milestone…

1 Like
24

Thanks for linking all of those, its good to know exactly what people mean in these discussions.

Yes probably there are problems with Julia Base from being both comprehensive and internally composable more than is often possible. Your problem of the reversed filter iteration is again that we can compose too many things together. It would be interesting to know if you can do anything like that at all in other languages.

With your issues with sum! etc over aliasing arrays, I think I assumed that or anything like it would break somehow or be undefined. But I can see that this would seem pretty weird if you expected otherwise:

julia> x = [1, 2, 3]
3-element Vector{Int64}:
 1
 2
 3

julia> sum!(x, x)
3-element Vector{Int64}:
 0
 0
 0

julia> y = [1, 2, 3]
3-element Vector{Int64}:
 1
 2
 3

julia> sum!(x, y)
3-element Vector{Int64}:
 1
 2
 3

Given I assumed that wouldn’t work, I wouldn’t personally class it as a huge correctness bug - so I’m not sure how clear the definitions we are using are.

3 Likes
25

Within this link:

“I don’t think it needs to be release blocking, as the bug has been open for years without too many people noticing.”

I appreciate that this specific example adienes linked to is being fixed, but it’s comments like these that do not make me optimistic of a cultural shift.

9 Likes
26

well, no, and that’s the point. I think reverse(::Filter) should be entirely removed :slight_smile: and reverse(::Zip) should error unless all of its constituent iterators are known to have the same finite length

6 Likes
27

Totaly. As I was getting at many of these bugs exist because you cant do these things in other languages - and that makes comparisons with other languages difficult.

Its also a problem to fix because removing it is a breaking change, and some people may productively use reverse(::Filter) with pure iterators!

As you mentioned we can hope that analysis can catch these at some point, but its hard. Maybe a docs warning is the best we have in the short term.

28

Is there any way to make Julia “bounty” issues ( ~related to correctness ) more visible to the community?

  • Where can I contribute to support a particular JuliaLang issue bounty?
    • adding my 10 - 100 - … $
  • Where can I see the existing bounties?
  • Is it possible to integrate with Github issues? For example, can we create a “Bounty” label ( in the JuliaLang/julia repo ) to highlight its importance?

For reference, “bounty” based development is emphasized in the tinygrad repository - as a “bounty-locked” label:

3 Likes
29

I mean, that seems like a totally unholdable claim. Nothing about “lazy filtering object” is unique to Julia, so why should it not be possible or be difficult to do in another language? To be quite honest, this just sounds like dodging the issue of “we didn’t think about that failure case” (which I take to be the core issue Yuri was originally getting at - undocumented/unexpected but quite obvious failure & edge cases).

The Julia semver stability guarantee has pretty much always been “if it’s in the manual, it’s API”. I’d be VERY surprised if this thing actually existing was guaranteed. That doesn’t change the fact that people may expect it to exist, but that’s precisely the issue - what is actually part of semver and supposed to work is not at all well defined in Julia today, leading to the sorts of correctness issues/unexpected bugs that people complain about all the time (rightly so!).

No amount of static analysis can compensate for a lack of documentation of core functionality, or contradicting documentation about semantics in type hierarchies.

and these are just some of the ones I personally filed. I’m not in the habit of keeping a list of these kinds of issues, but I do try to file them when I stumble across them.

14 Likes
30

Yup, and that’s easy to audit:

https://github.com/search?q=org%3ASciML+sum!&type=code

https://github.com/search?q=org%3ASciML+prod!&type=code

https://github.com/search?q=org%3ASciML+map!&type=code

All of our sum!, prod! (not even used), and map! look fine. And the uses of reverse! are on Array or Tuple with no Filter, so that seems fine.

https://github.com/search?q=org%3ASciML+reverse&type=code&p=1

Of course it should be improved anyways, but this is why I object to the phrase of “packages”. This shows about >100 of some of the most used packages in Julia do not touch the aforementioned issues. prod! doesn’t even show up in the source code. One should qualify what exactly is the issue in every instance, because many areas of Julia are doing fundamentally different things.

But not only that, qualifying what exactly is the issue is a constructive way to help get it fixed.

This is because those languages don’t include their major math libraries. numpy and PETSc have all sort of bugs and you hear about them during conference talks all of the time. The large the support surface the longer it takes to stabilize. Python is in a particularly weird position where its standard libraries tend to be so gimped that people tend to use many standard library replacements, which tend to be less supported and have quite a few bugs, and so I’m not sure how that’s any better.

Notably, the built in password and login libraries in Python are known to have security issues so they aren’t recommended and it’s been this way since 2008 or something like that. You’re taught as a beginning user to just avoid those modules, so therefore it’s safe? :person_shrugging: So people will define Python as safe by simply moving the boundary to not include those modules (which may end of life in 2028). Julia just has a different looking standard library since it’s historically been focused towards different developers, but in just the same way as you go further into the weeds you’ll find things less supported.

The larger the surface, the more there is to do.

22 Likes
31

Maybe I just got used to such quirks in dynamic languages, as all of them seem to have issues where silently wrong results can be returned. Here one example from R and Python each, that had bitten me in the past:

  • R: Recycling of vectors with non-matching length
  • Python: Before numpy made the switch to @, it had W * x mean matrix multiplication when W is a Matrix, but element-wise multiplication when W is an Array.

Just for fun, I checked the reverse filter example using std::ranges in C++20. Using the stateful predicate it compiles fine and then segfaults.There are very few compilers out there that warn about impure functions – mainly Haskell and some experimental effect systems. Basically, I only trust the Haskell compiler to a good extend (have not tried Rust though) – C++ is a lost case (compiled or not).

12 Likes
32

that’s interesting, thanks for trying it! still, I would prefer a (loud) segfault over getting the wrong answer

2 Likes
33

Me too, but unfortunately we don’t always get it. At least in my experience, Julia has not been worse than other dynamic languages in that regard.
I’m not saying these issues should not be fixed though … the less quirks and corner cases the better. Not just for that reason, I like the principled approach of Haskell stating laws that have to hold for its generic types, i.e., type classes. Guess that also Julia could benefit if required properties of its implicit interfaces could be easily tested, e.g., using PropCheck or something similar.

1 Like
34

That’s an API quirk, not a correctness bug – R is designed to do that intentionally. The Julia examples like reverse(zip( are incorrect implementations of documented behavior. Having a compiler check the documented assumptions is a nice feature but it’s not very useful if the implementation assumptions aren’t actually part of the documented interface in the first place.

3 Likes
35

The problem with the topic of Julia and correctness was how it lumped many issues with varying degrees of relevance into One Thing newcomers have to consider. It’s not. Newcomers care about whether they can write practical code that gives the correct results; yes they can, which is why we get more of the subsequent questions of “how do I optimize this”. Newcomers care about whether they can pick up packages that work properly; yes they can, and like any other language it depends on how actively maintained the package is.

It’s not particularly helpful for newcomers to be bombarded by bugs in corner cases or composition with little-used packages. For example, while I do agree that aliasing should be handled instead of silent, we know not to alias with reducing operations in practice, and we would never reduce over none of the dimensions, specifically sum!(copy(a), a) gives the same result as sum(a;dims=()) or copy(a). This is irrelevant to a newcomer who wants to Do Something. I get that people can be frustrated by problems in areas with little attention or demand, but that’s an issue for any language and it shouldn’t be a reason to avoid a language that is proven useful.

19 Likes
36

Technically correct, but it has a similar effect to program rather defensively when using it.

reverse(zip( I would also expect to work. filter with an impure predicate probably not, but it does not seem to be documented what can be assumed in that case.

37

If you look at the example in the issue, the mutable functor in question is user-defined:

And passed into Iterators.filter and Iterators.reverse.

As I’ve mentioned above statements like “seems like a totally unholdable claim” don’t help this conversation… we do actually want to know in what language you can both

  1. do that easily
    and
  2. it correctly handles the distinction between a pure function and an impure function

Then we have a clear comparison to make, and maybe something to learn from

38

I thought that the thing with composability was that reverse(filter(...)) would work without there being any explicit such method.

So either this particular example is not a composability issue, or you can’t really remove this, except by writing a method saying it’s not implemented (oh, and do those cause the most deliciously discombobulating error messages).

39

45 posts were split to a new topic: On the behavior of reversed iterators

85

This is becoming a post on reverse, perhaps this post should be split?

5 Likes