What does @inbounds actually mean?

johnomotani
Tue, 08 Aug 2023 08:39:14 +0000
Performance

In the discussion on https://github.com/JuliaLang/julia/issues/48245 it was mentioned that

Just to be clear, the reason we want to remove it is it requires us to compile the code more conservatively, leading to significant losses in inference accuracy, leading to significant losses of performance when running with check-bounds=no

and it was mentioned later in the thread that @inbounds is the same, even though using @inbounds more was the recommendation for replacing --check-bounds=no.

Could someone please explain what exactly @inbounds (and --check-bounds=no) is supposed to mean? As a scientific HPC user, I would expect that if I mark some piece of code with @inbounds that means:

assume that all array accesses are inbounds

therefore omit any bounds-checks

if input is given that does result in out-of-bounds access, undefined behaviour occurs (and the developer/user accepts the responsibility for ensuring this does not occur when using @inbounds)

I’d also expect that for any operation that did not produce an error without @inbounds, I would get exactly the same result (and type) when using @inbounds, while any operation that would produce an error with @inbounds is assumed not to occur. So I don’t understand why @inbounds would ‘requires us to compile the code more conservatively’ (as in the quote above) or mean (as mentioned elsewhere in the linked thread, if I understood correctly) that the type inference machinery cannot infer the return type of an operation with @inbounds that it could infer without @inbounds. Surely when without @inbounds there would be no error then the return type is the same, and by assertion this is the only case that needs to be handled, so the type is known at least as well as when not using @inbounds.

Or maybe another way of saying the same question is that the behaviour of an operation using @inbounds is unsafe, and allowing that is what @inbounds is for, but what does that have to do with type inference? If an array access inside an @inbounds block is out of bounds, the result is undefined behaviour. The undefined behaviour cannot be handled, but it seems like the statement is that type inference fails because in case of undefined behaviour, the compiler doesn’t know the return type (?). That statement seems illogical to me, because as I understand the intent of @inbounds, it tells the compiler (among other things) to assume that the access is always in-bounds, so the return type can be assumed to be whatever it is for an in-bounds access.

Since @inbounds (at least since julia-1.9.0) does affect type-inference, I guess something about my understanding of what it is supposed to mean is wrong! The docs only mention eliding bounds-checks to improve performance, not any potential for issues with type-inference.

9 posts - 4 participants