Soft scopes & globals in `@testitem`s

General Usage
,
1

I just encountered the problem discussed in Best solution to Julia's soft scope problem? , but in the context of unit tests. Specifically, in code like this:

@testitem "foo" begin
    A = 0
    for i in 1:10
        A += i
    end
    @test A == 55
end

This succeeds when running tests via pkg> test, but fails when running in VS Code’s test pane. Presumably because they differ about whether they create a new scope for each test?

Should this be considered a bug in whatever the test pane is using behind the scenes (part of the Julia Extension, I think?)? Or do test writers need to worry about scope problems like this, and sprinkle let and/or global declarations around? [I would advocate for not the latter.]

I have v1.120.2 of the Julia Extension installed.

2

Do you mean @testset rather than @testitem?

The following works for me whether I’m in the REPL (soft scope) or running it as a script julia foo.jl (hard scope), so I don’t see how soft-scoping is relevant here:

using Test
@testset "foo" begin
    A = 0
    for i in 1:10
        A += i
    end
    @test A == 55
end

(The reason it works is that @testset wraps everything in a let block, which creates a new scope, so A is not a global variable.)

3

I do mean @testitem, as is used for TestItemRunner or ReTestItems; but more specifically in this post, the error happens in the VS Code testing pane, whose details I don’t understand very well (see this thread: How does the Testing pane work?).

4

I wonder if anyone has further thoughts about this issue?

5

If the scopes for those macros don’t work like @testset, I would report it as a bug in the package(s) that define the macro.

Writing macros is tricky!

6

I reported the scope difference in Follow useful @testset behavior · Issue #51 · julia-vscode/TestItemRunner.jl · GitHub, but it didn’t really get any support.

7

If they don’t want to implement it, it’s up to them, but you could also probably get the behavior you want with

@testitem "foo" let
    # ...
end

instead of begin

8

Unfortunately, I don’t think this works at all.