Since you wrote this 15 hours ago, and I believe this is central to the actual post and initial discussion, this thread has sort of diverged a bit. But please do understand, that your characterization of the discussion could not be further from the truth. On the contrary, this has been discussed at length here on discourse (you can search for the thread that lead to SaferIntegers.jl for example), on github and on the old google mailing list. It has been discussed many times over, and hundreds, if not thousands, of developer hours have been spent thinking, replying, experimenting, concluding, and making a decision.
So please, do approach this with the open mind that you accuse existing Julia users and developers of lacking. We’ve talked about speed, accuracy, and “safety” in this thread, but to me this is actually not the central element we need to focus on. Realize that types of all variables involved matter for behavior, because multiple dispatch is used so much in Julia. Then you’ll perhaps, slowly at first, begin to appreciate why we don’t always “just convert to float”, “just check always” or similar. It gives your control, but you of course have to learn how things work to do this efficiently.
Now, it is possible to do something, such as introducing checking some strategic places, or changing lowering, but you simply just have to accept, that you’ll never get your will completely with respect to this issue. You seem to be blowing this up to something worth dismissing Julia over - if that’s correct, then I think you’re too focused on this battle over the raging world war around you
Something went wrong with the discourse quoting because I had apparently started another reply yesterday. I was quoting Jean, not you. So I’m not sure where you’re going with this.
When I wrote this characterization of the discussion it had not yet started, somehow. Now some facts about the design of Julia have emerged which put a better light on the design, even if it has not been consistently pursued:
Going for utmost speed and no safety by default, julia is never going to check default integers
OK for me, but then one should teach beginners about the existence of SafeInts so they can get security if they want.
Because we don’t care about them, Rationals are checked for overflow
This does not seem very consistent to me, and I would like Rationals to be respected a bit more. As a mathematician, for the kind of problems I study, I have no use at all for floating point but Rationals are very important. I would prefer for consistency that Rationals be optimized to the most and have no overflow checking; we can always make a Rational{SafeInt} for this latter purpose.
I am not sure that I understand the argument for consistency here — if you prefer overflow checking, but can’t get it for Int by default, why would you want to remove it from Rational which is apparently more important to you, just for “consistency”?
The problem with rationals is that you can overflow very, very easily, almost by default even if you do anything moderately complicated. And they are slow-ish anyway (compared to Int and Float64), so the marginal slowdown is usually acceptable.
Not doing Integer overflow checking is really not about speed vs. performance, doing so is infeasible. It’s like shooting yourself in the foot 5 times just so you don’t have to clip your toenails.
It may catch someone by surprise and this is a valid downside, but luckily these errors are easy to spot by unit testing and don’t typically occur in everyday programming (even in most scientific domains).
Maybe it would be a good idea to mention integer overflow also at another place in the manual, as people might just skip over the integer and floating-point numbers section?
For consistency there should exist a Rational type with no overflow checking: if overflow checking destroys speed as much as some committers mentioned, then certainly it destroys the speed of Rationals also. So to give proper respect to users of Rationals, one should allow them also to have the fast-and-dangerous version. As I said, the checked version can always be made by using Rational{SafeInt}. In the current situation, it is the non-checked version which cannot be made.
Calculations with Rational are already slow in the first place (comparatively).
I don’t see how this is about respect. You seem to be missing the fact that
silently overflowing Rationals are practically useless since you run into overflows almost trivially, and the calculation was not fast in the first place so the overflow checking is worth it, while
for Int & friends, you can do a lot of useful things without worrying about overflow.
This is all about practical usage considerations and technical arguments, not some nebulous concept of “consistency” or “respect”.
Of course, there is nothing preventing you from writing the package RationalsLivingDangerously.jl, with a
struct FootGunRational{T}
num::T
denom::T
end
and just borrowing all the code from Base sans overflow checking.
Rationals are already slow, and you won’t make them any faster by removing overflow checking. The reason they are slow is because you have to handle the numerator and denominator separately, not because they are poorly implemented.
The Rational type has a very focused scope, and in almost all cases a simple Float should be used instead.
I have many interests, and mathematics is certainly one of them!
If you have a use case for Rationals that’s perfectly fine, it’s a good type and the people who implemented it knew what they were doing.
If your current code using some Rationals is not as fast as it needs to be I’d suggest opening another thread with a small example. Who knows, maybe someone got a nice idea!
This is an uncharitable mischaracterization of what I’ve written, so I’m going to step out of the conversation after this. But I thought I’d offer one last attempt to explain our actual philosophy, more for those who are reading than for this reply which seems to be willful in its misunderstanding. All performance versus safety tradeoffs have been carefully considered. We prefer safety where it can be had at a reasonable cost. If the cost is too high, then performance wins.
If, in the future, hardware and compiler technology get to a point where we can do checked integer arithmetic everywhere without paying a terrible performance cost for it, we would very much consider making that the default (in a major release). It’s already been amply shown that rational int arithmetic without overflow checking is very dangerous (and not meaningful), whereas unchecked modular integers are so useful that even if we could do checked arithmetic by default we’d want to still expose that as well. Checking rationals and not checking ints is only superficially inconsistent if you completely ignore any rationale for the difference.
If you think the documentation of this could be improved, please make a pull request. This is open source, that’s how things get better.
There are good reasons for the way things are but there should be a convenient and well documented way of accomplishing what you hope to do.
Because what you’re doing is unique to many users’ daily workflow it may be worth having a setup (set of introductory materials/packages) that everyone trying to do this sort of thing knows to go to.
Every language has to do this sort of thing to some extent. If you’re doing any sort of matrix manipulation in Python you know to use numpy. If you’re using R you know that you’ll have to read through all of the formula documentations to fit models. If you’re using Matlab you know you’ll have to read all those bills to pay for it.
The point is that no language allows all users to just walk in and do whatever they want without paying their dues, but it may be harder if the infrastructure for doing that is not yet in place. If this is the case here, we need people like you to come in and help put those things together while keeping in mind that there may be unique technical issues you have to work with.
This has already been explored fairly extensively and the major speed issue with rationals is reducing the terms on every operation, which is so expensive that doing a bit of overflow checking is negligible. Tim Holy has a Ratios package that explores improving this, bit as the readme says, it has to make some sacrifices.
One idea would be to enable checked arithmetic during unit testing, or have some global flag in your project indicating whether you’re running in production mode or not, and use checked arithmetic if not.
I’m sure (and I really hope) there are better ways of doing this, but here’s a simple proof-of-concept:
function enable_checked_exp()
@eval Base begin
function ^(a::T, b::T) where {T<:BitInteger}
r1 = invoke(^, Tuple{Integer, Integer}, a, b)
r2 = BigInt(a) ^ BigInt(b)
r1 == r2 ? r1 : throw(OverflowError("$a^$b: got $r1, expected $r2"))
end
end
end
function gracefully_disable_checked_exp()
atexit(() -> run(`julia -q`))
exit()
end
I’ve advocated previously for such global solutions to problems like this (where you want different defaults), and there was always strong pushback because global side effects are evil (and I can certainly emphasize). Macros can sometimes be used to achieve this more cleanly, e.g. https://github.com/stevengj/ChangePrecision.jl
Yes, I already posted some excerpts and indeed I can read the code. The question which is not so easy to answer without knowing the intent of the programmer is:
can anyone confirm that Rationals are guaranteed to be checked for overflow for any single operation one can do on them?
