What does @inbounds actually mean?

General Usage Performance
,
1

In the discussion on https://github.com/JuliaLang/julia/issues/48245 it was mentioned that

Just to be clear, the reason we want to remove it is it requires us to compile the code more conservatively, leading to significant losses in inference accuracy, leading to significant losses of performance when running with check-bounds=no

and it was mentioned later in the thread that @inbounds is the same, even though using @inbounds more was the recommendation for replacing --check-bounds=no.

Could someone please explain what exactly @inbounds (and --check-bounds=no) is supposed to mean? As a scientific HPC user, I would expect that if I mark some piece of code with @inbounds that means:

  • assume that all array accesses are inbounds
  • therefore omit any bounds-checks
  • if input is given that does result in out-of-bounds access, undefined behaviour occurs (and the developer/user accepts the responsibility for ensuring this does not occur when using @inbounds)

I’d also expect that for any operation that did not produce an error without @inbounds, I would get exactly the same result (and type) when using @inbounds, while any operation that would produce an error with @inbounds is assumed not to occur. So I don’t understand why @inbounds would ‘requires us to compile the code more conservatively’ (as in the quote above) or mean (as mentioned elsewhere in the linked thread, if I understood correctly) that the type inference machinery cannot infer the return type of an operation with @inbounds that it could infer without @inbounds. Surely when without @inbounds there would be no error then the return type is the same, and by assertion this is the only case that needs to be handled, so the type is known at least as well as when not using @inbounds.

Or maybe another way of saying the same question is that the behaviour of an operation using @inbounds is unsafe, and allowing that is what @inbounds is for, but what does that have to do with type inference? If an array access inside an @inbounds block is out of bounds, the result is undefined behaviour. The undefined behaviour cannot be handled, but it seems like the statement is that type inference fails because in case of undefined behaviour, the compiler doesn’t know the return type (?). That statement seems illogical to me, because as I understand the intent of @inbounds, it tells the compiler (among other things) to assume that the access is always in-bounds, so the return type can be assumed to be whatever it is for an in-bounds access.

Since @inbounds (at least since julia-1.9.0) does affect type-inference, I guess something about my understanding of what it is supposed to mean is wrong! The docs only mention eliding bounds-checks to improve performance, not any potential for issues with type-inference.

2

From the thread you’re referencing, it appears that @inbounds (and --check-bounds=no) prevents constant folding. E.g. in loops with literal or constant bounds the compiler can in some instances run the whole loop and just emit the result. So it eliminates the whole loop (at the cost of longer compilation time). For some reason @inbounds prevents this from happening. It could be that other subtle side effects can appear as well?

3

“Inference” in that thread is not necessarily referring to inferring the return type exclusively. A large part of the inference mechanism is about inferring the sideeffects of julia code, like array accesses that could be out of bounds, whether all memory that’s accessed is allocated in the function in question (so no memory from outside is used), whether the function can throw any error, whether the function is known to terminate (so no loops of unknown length) - that kind of stuff. This is the part of inference that is influenced by @inbounds and --check-bounds=no. The return type of the function is still inferred just fine in these cases! What can happen though is that there are cases where the compiler, if it is allowed to do bounds checking itself, is able to prove that a loop cannot throw an error, array accesses are always inbounds and thus the compiler can compute the result of the loop at compile time, and insert it as a constant into the function, instead of making you compute the result at runtime. Conversely, with things like @inbounds and --check-bounds=no, the compiler cannot prove that, because it’s not allowed to; after all, using @inbounds and --check-bounds=no explicitly instructed the compiler not to do that. It’s thus forced to insert the loop, leading to worse overall runtime performance.

This does simplify the issue a bit, but that’s the general idea.

2 Likes
4

@sgaure, @Sukera I don’t think the problem is only with loops. I started asking this question because of a slow down in Makie.jl when I used --check-bounds=no (Slowdown when using `--check-bounds=no` on julia-1.9.x · Issue #3132 · MakieOrg/Makie.jl · GitHub) and managed to trace the issue there down to a minimal example (const-propagation with `isabstracttype()` fails when using `--check-bounds=no` in 1.9.x · Issue #50825 · JuliaLang/julia · GitHub) that’s just a function containing an if-statement.

5

FYI, that specific issue is unrelated to --check-bounds=no, and has been fixed on master already.

1 Like
6

Thanks @Sukera! Sounds like it was just a bug that was confusing me then :slight_smile:

7

That’s a bit weird. Those options are sold as performance enhancers, i.e. from the documentation it seems it’s just removing the checks on indices, to save some cycles. In particular when the @inbounds occurs on a single statement inside the loop, it suggests that the only thing happening is that a check on the indexing in that statement is removed, nothing else. Compilers normally look for several transformations of loops, like blocking, changing the order etc. Are some such transformations prevented as well?

8

With @inbounds, the compiler cannot check for out of bounds access in the enclosed region independently; it must assume that the access is inbounds. This can lead to the compiler assuming that a loop is safe, even if it could otherwise prove that there is a circumstance where you’d get an out of bounds access. The docstring says as much, by imploring the user to check that the use of @inbounds is safe to do.

I’m aware that they’re sold as performance enhancers, but that’s mostly because people think of bounds checking as a pesky performance overhead when compared to languages that don’t do bounds checking by default (most commonly C/C++). The reality is that using that “performance enhancer” (without appropriate checks!) is ignoring both correctness and safety of the code in question. It’s unfortunate but unavoidable that the abstraction of low level details like pointer accesses then leak when the bounds checking would have prevented an issue.

To be clear, I also don’t think @inbounds should mention that this is for performance (though that’s pretty much alwas the goal when it is used correctly & naively).

9

The fundamental difference isn’t inbounds specific. The problem is that with @inbounds running the code with out of bounds accesses produces undefined behavior (which can include crashing your program). As such, for the compiler to run code that contains @inbounds at compile time, the compiler must be able to prove that the bounds it will access are all in bounds. without @inbounds The compiler can just run the code in a try/catch and know that running your code won’t crash the compiler.