# Debugger doesn't stop at breakpoints when running tests (but does during normal execution)

**URL:** <https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202>\
**Category:** VS Code\
**Tags:** testing, debugging, vscode\
**Created:** [January 19, 2023, 9:14am UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202 "2023-01-19T09:14:15Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![eott-siz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eott-siz/32/44936_2.png) [@eott-siz](https://discourse.julialang.org/u/eott-siz)\
**Post date:** [January 19, 2023, 9:14am UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/1 "2023-01-19T09:14:16Z")

</div>

Julia 1.7.3  
VS Code 1.74.3  
Julia extension 1.38.2

I’m having a problem where, when I run tests with the debugger on the program I’m working on, the debugger doesn’t stop at any breakpoints I set. If I run the program normally (meaning executing a `main` function) with the debugger, the debugger works as expected and stops. I tried isolating the problem with a small example problem I can share here, however I can’t get the problem to reproduce in the example. Until I do, I suppose my question is which things in particular might be causing this issue and what I should look into.

## Example

In the following the example and how I use it. I think it captures the structure and use of the full program well, but evidently I’m missing something.

```julia-auto
project_dir
|> deps
  |> build.jl (empty)
|> src
  |> ModuleToTest.jl
  |> PackageToTest.jl
|> test
  |> runtests.jl
|> Manifest.toml
|> Project.toml

```

ModuleToTest.jl

```julia
module ModuleToTest

export do_something

function internal_only(x)
    return x + 1
end

function do_something(x)
    result = internal_only(x)
    return result
end

end

```

PackageToTest.jl

```julia
module PackageToTest

include("ModuleToTest.jl")
using .ModuleToTest

function main()
    print("The magic number is $(ModuleToTest.do_something(41))")
end
main()

end

```

runtests.jl

```julia
using Test
using PackageToTest.ModuleToTest

@testset "tests_are_working" begin
    @test true
end

@testset "test_do_something" begin
    @test ModuleToTest.do_something(41) == 42
end

@testset "test_internal_only" begin
    @test ModuleToTest.internal_only(41) == 42
end

```

Project.toml

```julia
name = "PackageToTest"
uuid = "2e0f99b1-0d8f-4296-8d0c-1c50a50d04c8"
authors = ["author"]
version = "0.1.0"

[deps]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

```

## Running the example

For debugging the program and testing it I have two debugger settings in `launch.json`:

```json
{
    "type": "julia",
    "request": "launch",
    "name": "Run file",
    "program": "project_dir/src/PackageToTest.jl",
    "stopOnEntry": false,
    "cwd": "project_dir/",
    "juliaEnv": "${command:activeJuliaEnvironment}",
    "args": []
},
{
    "type": "julia",
    "request": "launch",
    "name": "Run tests",
    "program": "project_dir/test/runtests.jl",
    "stopOnEntry": false,
    "cwd": "project_dir/",
    "juliaEnv": ".",
    "args": []
}

```

The launch config `Run file` runs the program with its `main` function, and will stop at breakpoints. Launching with `Run tests` will stop at breakpoints in this example, but not in the full project. I don’t use the REPL other than to add dependencies to `Project.toml`. Running tests from the PKG REPL, by `activate .` then `test PackageToTest`, runs the tests and works as normal, but I haven’t figured out how to use the debugger in the REPL for tests.

---

<div class="post-metadata">

**Author:** ![eott-siz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eott-siz/32/44936_2.png) [@eott-siz](https://discourse.julialang.org/u/eott-siz)\
**Post date:** [January 26, 2023, 8:41am UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/2 "2023-01-26T08:41:48Z")

</div>

An update to our problem. It’s not solved, but I do have a new avenue to pursue. Let’s change the example by moving tests into other files that are being included:

runtests.jl

```julia
using Test

@testset "tests_are_working" begin
    @test true
end

include("included_tests.jl")

```

included\_tests.jl

```julia
using PackageToTest.ModuleToTest

@testset "test_do_something" begin
    @test ModuleToTest.do_something(41) == 42
end

@testset "test_internal_only" begin
    @test ModuleToTest.internal_only(41) == 42
end

```

With this modification the example reproduces our problem. With launch config `Run tests`, the debugger will stop at breakpoints in file `runtests.jl` but not at breakpoints in any other file that is on the code path of tests, e.g. `ModuleToTest.jl` or `included_tests.jl`.

This is now beginning to sound similar to problems other users have found and solved, so I’m hoping I will stumble over a solution there.

---

<div class="post-metadata">

**Author:** ![nrontsis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nrontsis/32/38535_2.png) [@nrontsis](https://discourse.julialang.org/u/nrontsis)\
**Post date:** [February 11, 2023, 12:31pm UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/3 "2023-02-11T12:31:19Z")

</div>

I have run to what I think were similar issues in [this thread](https://discourse.julialang.org/t/how-to-debug-exception-triggered-by-unit-test-in-vscode/90438/2)

---

<div class="post-metadata">

**Author:** ![eott-siz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eott-siz/32/44936_2.png) [@eott-siz](https://discourse.julialang.org/u/eott-siz)\
**Post date:** [February 13, 2023, 9:07am UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/4 "2023-02-13T09:07:37Z")

</div>

It seems possible the problems are related, yes. Unfortunately I don’t have much new to offer as solutions go.

Something that might be related is the “compiled mode” of the debugger. Apparently when enabled it won’t stop at breakpoints (and, perhaps, exceptions) in code files other than the file being executed. So, disabling this mode sounds like something that could solve our issue. I’d love to test it, but I cannot _enable_ compiled mode. I don’t think the feature is broken, so it must be a problem with how we use environments and execute code / tests.

---

<div class="post-metadata">

**Author:** ![nrontsis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nrontsis/32/38535_2.png) [@nrontsis](https://discourse.julialang.org/u/nrontsis)\
**Post date:** [February 14, 2023, 4:31pm UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/5 "2023-02-14T16:31:40Z")

</div>

For what it’s worth, I have found `Infiltrator.jl`’s `@infiltrate` to work even within `@testset` and to be very helpful for debugging tests (in combination with `@exfiltrate` if necessary).

---

<div class="post-metadata">

**Author:** ![eott-siz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eott-siz/32/44936_2.png) [@eott-siz](https://discourse.julialang.org/u/eott-siz)\
**Post date:** [February 16, 2023, 9:43am UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/6 "2023-02-16T09:43:38Z")

</div>

I tried `Infiltrator.jl` and while it seems useful, it leads to a much more cumbersome workflow and has limited stepping functionality. I will keep it in mind for when the speedup through compile mode becomes necessary.

However, I also tried using `Debugger.jl` in the REPL directly without going through the VS Code plugin. In the example above this would be by:

PKG REPL:

```julia
activate .
add Debugger

```

Julia REPL:

```julia
using Debugger
@enter include("test/runtests.jl")

```

This will stop in the `include` function and open the Debugger REPL:

```julia
bp add "./src/ModuleToTest.jl":6
c

```

This should stop at the line of `internal_only`’s definition when it is called during the tests, however it doesn’t. What I take from this is that the problem lies with `Debugger.jl` (or more likely with how we use it) and has nothing do with the plugin’s GUI for the debugger. Once I have more information I’ll take it over to the appropriate place for that.

---

<div class="post-metadata">

**Author:** ![nrontsis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nrontsis/32/38535_2.png) [@nrontsis](https://discourse.julialang.org/u/nrontsis)\
**Post date:** [February 22, 2023, 8:50am UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/7 "2023-02-22T08:50:19Z")

</div>

[This](https://discourse.julialang.org/t/why-is-test-so-meta-programming-heavy/94619/32) is also related, I believe. It appears that the authors of the Julia VSCode plugin are already aware of the limitations in `debugging` while in `@testset`s, and focus is on the [new testitem framework](https://discourse.julialang.org/t/prerelease-of-new-testing-framework-and-test-run-ui-in-vs-code/86355).

---

<div class="post-metadata">

**Author:** ![eott-siz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eott-siz/32/44936_2.png) [@eott-siz](https://discourse.julialang.org/u/eott-siz)\
**Post date:** [February 22, 2023, 9:50am UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/8 "2023-02-22T09:50:06Z")

</div>

I have come across the TestItem framework while looking for solutions, but haven’t found the time to check if switching to it might be worth the effort. I have already invested too much time in this, but will earmark it for future reference.

Meanwhile, as I suspected the underlying issue lies with a limitation in `Debugger.jl` and [a solution of sorts](https://github.com/JuliaDebug/Debugger.jl/issues/328) was found that allows tests to be debugged. The debugger is intended to be used on functions, not on what is essentially deeply nested top-level code. So wrapping testsets into functions and calling them within the actual @testset macro, still works for the regular test executions, but also allows for debugging individual tests by calling them via `@enter test_to_be_debugged()` from the REPL.

However, I haven’t yet figured out how to connect a running REPL to the debugger panel in VS Code yet, which I want to do so I can use the GUI it provides for inspecting state at a breakpoint. Once I find out how, or find out it isn’t possible, I’ll update again and mark this as a solution. The underlying debugger issue is solved, but not the problem when using it from the VS Code plugin. I’ll be honest, I still do not care for the REPL.

---

<div class="post-metadata">

**Author:** ![eott-siz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eott-siz/32/44936_2.png) [@eott-siz](https://discourse.julialang.org/u/eott-siz)\
**Post date:** [March 8, 2023, 12:48pm UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/9 "2023-03-08T12:48:39Z")

</div>

An (hopefully final) update to the original question of how to debug tests with the debugging functionality within the VS Code plugin for Julia:

First, as described in the previous post, let’s modify our tests so that the code lives within functions.

`included_tests.jl`

```julia
using Test
using PackageToTest.ModuleToTest

function test_do_something()
    @test ModuleToTest.do_something(41) == 42
end

@testset "test_do_something" begin
    test_do_something()
end

function test_internal_only()
    @test ModuleToTest.internal_only(41) == 42
end

@testset "test_internal_only" begin
    test_internal_only()
end

```

and we need a debug launch config that runs the active file (I think this is the default config):

`launch.jl`

```json
{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "julia",
            "request": "launch",
            "name": "Run active file",
            "program": "${file}",
            "stopOnEntry": false,
            "cwd": "projectdir/",
            "juliaEnv": "${command:activeJuliaEnvironment}",
            "args": []
        },
    ]
}

```

Now we can open file `included_tests.jl`, hit F5 and the debugger will stop at breakpoints in file `ModuleToTest.jl` and display the values of variables in the debug pane, as well as allowing for the stepping functions to work as expected.

---

<div class="post-metadata">

**Author:** ![MichielVanDerMeulen](https://avatars.discourse-cdn.com/v4/letter/m/9e8a1a/32.png) [@MichielVanDerMeulen](https://discourse.julialang.org/u/MichielVanDerMeulen)\
**Post date:** [July 19, 2023, 10:16am UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/10 "2023-07-19T10:16:45Z")

</div>

I’m trying to get this solution to work on my Windows setup in VSCode, but so far I have no success. I have completely reproduced the setup as suggested [above](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/9), but I cannot get the debugger to stop at breakpoints in the module code.

In fact, even a breakpoint placed inside the test function at the line where we test `...(41) == 42` is not picked up. I can get the debugger to stop when I put a breakpoint on the `test_internal_only()` call, but still I cannot step into the function:

```julia
function test_internal_only()
    @test ModuleToTest.internal_only(41) == 42
end

@testset "test_internal_only" begin
    test_internal_only()
end

```

The only way I’m able to actually step into the method under test is to move its evaluation outside of the `@test` macro, put a breakpoint at the `actual = ...` line and step in manually with F11:

```julia
@testset "test_internal_only" begin
    actual = ModuleToTest.internal_only(41)
    @test actual == 42
end

```

Still, the debugger doesn’t respect breakpoints in the method under test.

Ultimately, when I’m developing a bigger package I would like to be able to split up the test suite into smaller files and `include` them in the `runtests.jl`, but first I’d like the debugger be able to work on this more simple setup. Does anyone have a clue why this doesn’t work, or have a suggestion how to solve this in an alternative manner?

---

<div class="post-metadata">

**Author:** ![eott-siz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eott-siz/32/44936_2.png) [@eott-siz](https://discourse.julialang.org/u/eott-siz)\
**Post date:** [July 19, 2023, 12:39pm UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/11 "2023-07-19T12:39:04Z")

</div>

I just tried my own instructions (spread across three posts - something to improve) in a new workspace and folder and was able to replicate the correct behaviour when running launch config “Run active file” on file “included\_tests.jl”.

Julia v1.8.5  
VS Code v1.80.1  
VS Code Julia plugin v1.47.2

Of note is that I used a new julia environment\* and VS Code did ask me to switch my julia enviornments. I have to confess that I still don’t really environments, so I can’t say if that has any impact.

\* By doing:  
`julia ]`  
`activate .`  
`add Test`

---

<div class="post-metadata">

**Author:** ![MichielVanDerMeulen](https://avatars.discourse-cdn.com/v4/letter/m/9e8a1a/32.png) [@MichielVanDerMeulen](https://discourse.julialang.org/u/MichielVanDerMeulen)\
**Post date:** [July 19, 2023, 1:37pm UTC](https://discourse.julialang.org/t/debugger-doesnt-stop-at-breakpoints-when-running-tests-but-does-during-normal-execution/93202/12 "2023-07-19T13:37:53Z")

</div>

Ah! I somehow got your example to work as well, I think there was a mistake in the `launch.json` file (just removing it completely fixed it).

It would be a nice to have to also be able to debug from a main test script (like in your `runtests.jl` with one or more `include`s), but the above is very helpful to have at least something. Thanks 🙂