# How to test that bounds checks are happening

**URL:** <https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585>\
**Category:** General Usage\
**Created:** [November 21, 2022, 12:25pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585 "2022-11-21T12:25:22Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 21, 2022, 12:25pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/1 "2022-11-21T12:25:22Z")

</div>

When running CI, --check-bounds=yes is passed to julia as a startup option. How can I test to make sure I’m not misusing `@inbounds`?

In this example, my bounds checking is broken, but the test still passes:

```julia
x@X ~ % julia --banner=no --check-bounds=yes
julia> function bad(v, i)
           i < firstindex(v) && i > lastindex(v) && threw(BloundsError()) # bad
           @inbounds v[i]
       end
bad (generic function with 1 method)

julia> using Test

julia> @test_throws BoundsError bad([1,2,3], 4)
Test Passed
      Thrown: BoundsError

```

In a more real-world example, I want to test that a sorting function throws a bounds error when passed out of bounds indices rather than segfaulting. `@test_throws sort!(...)` will pass even when it shouldn’t.

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [November 21, 2022, 12:30pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/2 "2022-11-21T12:30:55Z")

</div>

Why should the test not pass? You are testing if the expression throws a `BoundsError`, it does, so the test pass.

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 21, 2022, 1:17pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/3 "2022-11-21T13:17:54Z")

</div>

How can I write a test that will fail in this case? How can I test to make sure that the BoundsError did not come from within an `@inbounds` block?

---

<div class="post-metadata">

**Author:** ![jakobnissen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakobnissen/32/13477_2.png) [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)\
**Post date:** [November 21, 2022, 1:23pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/4 "2022-11-21T13:23:04Z")

</div>

I would extend `Base.checkbounds(Bool, ::A, ::I)` with a new method, and test that directly.

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [November 21, 2022, 1:28pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/5 "2022-11-21T13:28:58Z")

</div>

Making a new AbstractArray type `NeverInboundsArray` which would override `getindex` to throw BoundsError (or another TestError) error? (i.e. always check bounds explicitly and then pass through to underlying array)

---

<div class="post-metadata">

**Author:** ![greg\_plowman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/greg_plowman/32/8100_2.png) [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)\
**Post date:** [November 21, 2022, 1:30pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/6 "2022-11-21T13:30:06Z")

</div>

throw and test for `ArgumentError` instead?

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [November 21, 2022, 1:44pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/7 "2022-11-21T13:44:26Z")

</div>

Not super elegant but you could do `throw(MyBoundsError())` in your own bounds check and define

```julia
MyBoundsError() = BoundsError()

```

Then in the tests you redefine it to something else which you can distinguish from `BoundsError`.

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [November 21, 2022, 2:41pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/8 "2022-11-21T14:41:18Z")

</div>

> [@jakobnissen](#):
>
> I would extend `Base.checkbounds(Bool, ::A, ::I)` with a new method, and test that directly.

What is `A`? In neither example am I defining a subtype of AbstractArray.

> [@Dan](#):
>
> Making a new AbstractArray type `NeverInboundsArray` which would override `getindex` to throw BoundsError (or another TestError) error? (i.e. always check bounds explicitly and then pass through to underlying array)

If NeverInboundsArray throws a BoundsError, then tests would pass (bad). If NeverInboundsArray throws a different error, then the tests above would fail (good), but this would also fail (bad)

```julia
function good(v, i)
    @boundscheck checkbounds(v, i)
    @inbounds v[i]
end
@test_throws BoundsError good(NeverInboundsArray([1,2,3]), 4) 

```

And this would pass regardless (bad)

```julia
function bad(v, i)
    i < firstindex(v) && i > lastindex(v) && threw(BloundsError()) # bad
    @inbounds good(v, i)
end
function good(v, i)
    @boundscheck checkbounds(v, i) # Throws BoundsError here when --check-bounds=yes
    @inbounds v[i]
end
@test_throws BoundsError bad(NeverInBoundsArray([1,2,3]), 4)

```

This is the best solution so far, but not great.

> [@greg\_plowman](#):
>
> throw and test for `ArgumentError` instead?

A BoundsError is most appropriate in this case, and because the error is user-facing I don’t want to compromise on its type.

> [@GunnarFarneback](#):
>
> Not super elegant but you could do `throw(MyBoundsError())` in your own bounds check and define…

If possible, I’d like to keep the source code elegant and do any necessary shenanigans only in testing. I’m worried it might be just as error-prone to do this as to leave the functionality untested.

* * *

To clarify, I’m looking to test that “This would throw a bounds error whether or not we are running Julia with `--check-bounds=yes`”.

To adapt @Dan’s answer,

```julia
function Base.getindex(a::NeverInBoundsArray, inds...)
    if !checkbounds(Bool, a.x, inds...)
        error = if @inbounds_is_active_in_current_scope
            SegfaultError()
        else
            BoundsError(a, inds)
        end
        throw(error)
    end
    @inbounds getindex(a.x, inds...)
end

```

would work, but I don’t know how to get the `@inbounds_is_active_in_current_scope` macro to work when Julia is run with `--check-bounds=yes`.

Alternatively, if Julia automatically converted `BoundsError`s thrown in `@checkbounds` checks that are in the scope of `@inbounds` into `SegfaultError`s or `ElidedBoundsError`s, then the code in the OP would be correct.

---

<div class="post-metadata">

**Author:** ![jakobnissen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakobnissen/32/13477_2.png) [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)\
**Post date:** [November 21, 2022, 2:49pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/9 "2022-11-21T14:49:24Z")

</div>

> What is `A`? In neither example am I defining a subtype of AbstractArray.

`A` is your type here. Although the docs for `checkbounds` mention “array”, I don’t think this function needs to be array-specific. Indeed, it’s also used for strings in Base.

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [November 21, 2022, 4:06pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/10 "2022-11-21T16:06:06Z")

</div>

> [@Lilith](#):
>
> ```julia
> if !checkbounds(Bool, a.x, inds...)
> error = if @inbounds_is_active_in_current_scope
> SegfaultError()
> else
> BoundsError(a, inds)
> end
> throw(error)
> end
> 
> ```

I think the `@boundscheck` macro would help. It marks a block as a boundcheck block and doesn’t compile when @inbounds is active. Something like:

```julia
if !checkbounds(Bool, a.x, inds...)
        error = SegfaultError()
        @boundscheck error = BoundsError(a, inds)
        throw(error)
    end

```

~~note: couldn’t get this to work normally on REPL.~~  
A little demo of @boundscheck in REPL.

```julia
julia> test(v) = begin
       @inline 
       error = BoundsError()
       @boundscheck error = SegmentationFault()
       throw(error)
       end

test (generic function with 1 method)

julia> test2(v) = @inbounds test(v)
test2 (generic function with 1 method)

julia> test2(rand(100)) # runs with @inbounds
ERROR: BoundsError

julia> test(rand(100)) # runs without @inbounds
ERROR: SegmentationFault()

```

Does this help solve the problem? Not if `--bounds-check=yes` since then the outputs are the same.

---

<div class="post-metadata">

**Author:** ![cjdoris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cjdoris/32/213133_2.png) [@cjdoris](https://discourse.julialang.org/u/cjdoris)\
**Post date:** [November 21, 2022, 4:32pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/11 "2022-11-21T16:32:23Z")

</div>

Do you actually just want `--check-bounds=auto`?

If you’re using the julia-runtest action in GitHub you can set `check_bounds: auto`.

---

<div class="post-metadata">

**Author:** ![milankl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milankl/32/4198_2.png) [@milankl](https://discourse.julialang.org/u/milankl)\
**Post date:** [April 19, 2024, 9:42pm UTC](https://discourse.julialang.org/t/how-to-test-that-bounds-checks-are-happening/90585/12 "2024-04-19T21:42:08Z")

</div>

Yes the `--check-bounds=yes` as a default for CI makes sense, but I guess there is no way to overwrite this for single annotations? In order to use CI to test that `@boundscheck` is compiled away with `@inbounds` I’ve set up some tests like

```julia
f(A, i) = @inbounds A[i]
@test f(A, j) == A[k]

```

because for my custom array the out of bounds index `j` points to the element at `k` which isn’t correct but convenient for testing. So I’d need a `@enforce_inbounds` that’s always `@inbounds` regardless the CI settings.