that’s interesting, thanks for trying it! still, I would prefer a (loud) segfault over getting the wrong answer
2 Likes
Me too, but unfortunately we don’t always get it. At least in my experience, Julia has not been worse than other dynamic languages in that regard.
I’m not saying these issues should not be fixed though … the less quirks and corner cases the better. Not just for that reason, I like the principled approach of Haskell stating laws that have to hold for its generic types, i.e., type classes. Guess that also Julia could benefit if required properties of its implicit interfaces could be easily tested, e.g., using PropCheck or something similar.
1 Like
That’s an API quirk, not a correctness bug – R is designed to do that intentionally. The Julia examples like reverse(zip( are incorrect implementations of documented behavior. Having a compiler check the documented assumptions is a nice feature but it’s not very useful if the implementation assumptions aren’t actually part of the documented interface in the first place.
3 Likes
The problem with the topic of Julia and correctness was how it lumped many issues with varying degrees of relevance into One Thing newcomers have to consider. It’s not. Newcomers care about whether they can write practical code that gives the correct results; yes they can, which is why we get more of the subsequent questions of “how do I optimize this”. Newcomers care about whether they can pick up packages that work properly; yes they can, and like any other language it depends on how actively maintained the package is.
It’s not particularly helpful for newcomers to be bombarded by bugs in corner cases or composition with little-used packages. For example, while I do agree that aliasing should be handled instead of silent, we know not to alias with reducing operations in practice, and we would never reduce over none of the dimensions, specifically sum!(copy(a), a) gives the same result as sum(a;dims=()) or copy(a). This is irrelevant to a newcomer who wants to Do Something. I get that people can be frustrated by problems in areas with little attention or demand, but that’s an issue for any language and it shouldn’t be a reason to avoid a language that is proven useful.
19 Likes
Technically correct, but it has a similar effect to program rather defensively when using it.
reverse(zip( I would also expect to work. filter with an impure predicate probably not, but it does not seem to be documented what can be assumed in that case.
If you look at the example in the issue, the mutable functor in question is user-defined:
And passed into Iterators.filter and Iterators.reverse.
As I’ve mentioned above statements like “seems like a totally unholdable claim” don’t help this conversation… we do actually want to know in what language you can both
- do that easily
and - it correctly handles the distinction between a pure function and an impure function
Then we have a clear comparison to make, and maybe something to learn from
I thought that the thing with composability was that reverse(filter(...)) would work without there being any explicit such method.
So either this particular example is not a composability issue, or you can’t really remove this, except by writing a method saying it’s not implemented (oh, and do those cause the most deliciously discombobulating error messages).
This is becoming a post on reverse, perhaps this post should be split?
5 Likes
At first wasn’t very concerned about this example. It’s typical of examples meant to illustrate the rampant incorrectness.
In general the function is called like map!(c, a, b). In the examples in the doc string, the input and output are clearly not aliased. In fact c is freshly initialized. Many numerical libraries work like this. It is very common to read something like “output must not be aliased to input”. So if I see the doc to map!(c, a, b) and thought, “well it didnt say I cant alias, I guess I just do it and move on”, I’m asking for trouble. If I didn’t read the doc string, I’d never assume it. I’d check it first or ask or something.
If this were the end of the story, I’d say it’s not such a big deal.
The bigger problem, apparently, is that
- behavior with aliasing is not documented.
- behavior with aliasing was not properly tested.
- behavior with aliasing changed in a (non-breaking) release.
Now there’s nothing I can do to avoid a bug in my code, even if I’m careful.
It’s possible that documenting it more carefully may have avoided this. It would have made it less likely that adequate tests were overlooked, and that a breaking change would be introduced.
Note that although the previous behavior has been restored, it’s still not documented.
More generally: I won’t put much stock in the idea that Julia is really buggy based on anecdotes. I’d have to see some attempt at an analysis. And discussion of which languages you’re comparing to.
EDIT:
10 Likes
I have long had a question related to this. I’m not in a position to let other people to use my code. All I write are very small and ad-hoc. But, some of my tools may grow enough to be interesting to other people in the future.
So, I wonder: Why am I writing for j in 1:size(arr,2); for i in 1:size(arr,1), assuming that the array index starts from 1 and is contiguous, even when I don’t need to assume that.
So, my question is, why don’t you recommend writing everything as general as possible if that doesn’t increase the size of the code too much and if that doesn’t reduce the capability of the functions you are writing?
So,
Using abstract types requires significantly more testing than using concrete types.
I wonder if you can elaborate on that? Do you mean that testing is more difficult even if you test only on the few concrete types that you would be anyway using if you wrote a concrete interface?
. . . Anyway, we need some textbook or document that describes “the best practices”. Everything about generality seems confusing.
1 Like
This is a great list @adienes, thanks. Care to try tackling some of these yourself? Those who notice problems from such issues are usually best-motivated to start fixing them. And thanks for the several pull requests, too!
6 Likes
It was just removed from the 1.10 milestone (see: sum!, prod!, any!, and all! may silently return incorrect results · Issue #39385 · JuliaLang/julia · GitHub), which I see as a sign of ignorance of leading team members for correctness issues…
I understand your frustration, but these are internal organizational tools that have very clearly defined processes and requirements. This does not mean the issue is not valued. It doesn’t mean that leading team members don’t care about correctness. It doesn’t mean it won’t get fixed. It doesn’t mean that its fix (or mitigation or …) won’t get into 1.10, either.
10 Likes
Stroustrup said that C++ protects you from accidents not sabotage. That should be applicable to most other languages, too.
2 Likes
There are currently 25 bugs in the issue tracker that have the label “correctness issue”. Two of them are on the list of the bugs to be fixed for 1.10., which is nice… But it also means we probably need to wait another 6 years or 12 releases until all of them are fixed…
1 Like
An easy, and pretty reasonable, solution to the aliasing “bug” would be to add notes to all such functions that “The mutated argument must not alias any other argument”. I personally wouldn’t really expect such a call that involves aliasing between arguments to behave in any particular way, unless explicitly stated in the docsting.
8 Likes
It seems that the issue has been open since January '21 - and the behavior was observed even in 1.5.3.
I wouldn’t expect the issue suddenly becoming a release-blocking issue.
However, some low-hanging fruits are more worrying: I understand the bug fix not being a release-blocking priority, but why not add the minimal documentation to help people avoid the issue in the meantime?
On the same note - it has been known for a while that @async is to be avoided in favor of @spawn to the extent that we have a clear warning against using @async: however, the new users reading the manual are clearly presented with learning material that goes against the documentation.
These easy-to-achieve goals that seem to reflect caring about new users are things that get me worried: the experienced users can get along and just avoid these traps (having access to a kind of special lore or simply being familiar to a larger extent with the up-to-date documentation), while new Julia users (who are primarily reading the manual before going into more depth) are actually taught to use the language in the wrong way.
9 Likes
Well, if it is open for 2.5 years it should get fixed finally, don’t you think?
If we are still not doing it we are ignorant for correctness issues, that is all I say…
Again, you’re misreading what it means to be (or not be) on a particular milestone release. Being placed on a milestone means that it’s a release blocker — that it must be fixed for the release. Not being on a milestone does not mean the converse.
6 Likes