# Different result of \`===\` between \`@runtest\` and REPL

**URL:** <https://discourse.julialang.org/t/different-result-of-between-runtest-and-repl/108924>\
**Category:** General Usage\
**Tags:** testing\
**Created:** [January 17, 2024, 7:30pm UTC](https://discourse.julialang.org/t/different-result-of-between-runtest-and-repl/108924 "2024-01-17T19:30:54Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![bthorne93](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bthorne93/32/19570_2.png) [@bthorne93](https://discourse.julialang.org/u/bthorne93)\
**Post date:** [January 17, 2024, 7:30pm UTC](https://discourse.julialang.org/t/different-result-of-between-runtest-and-repl/108924/1 "2024-01-17T19:30:54Z")

</div>

In an entirely clean package, the following unit test passes:

```julia
using Test 

@testset begin
    function build_matrix()
        A = zeros(2, 2)
        A[1, 2] = A[2, 1] = 1
        return A
    end
    
    A = build_matrix()
    B = build_matrix()
    @test A === B
    
end

```

Whereas, in the REPL, `A===B` returns `false`, which is what I would expect. What is going on here?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [January 17, 2024, 9:35pm UTC](https://discourse.julialang.org/t/different-result-of-between-runtest-and-repl/108924/2 "2024-01-17T21:35:31Z")

</div>

Hmm… I think it has to do with local scope, but I agree this is surprising.

Let me simplify the example.

```julia-repl
julia> let
           function build_matrix()
               A = zeros(2,2)
               A[1,2] = A[2,1] = 1
               return A
           end
           A = build_matrix()
           B = build_matrix()
           A === B
       end
true

```

What is happening is that the binding `A` refers to `A` within the local scope of the `let` block. The second time we call `build_matrix` it reassigns `A`.

```julia
julia> let
           function build_matrix()
               A = zeros(2,2)
               @info "build_matrix: " pointer(A)
               A[1,2] = A[2,1] = 1
               return A
           end
           A = build_matrix()
           @info "A = build_matrix()" pointer(A)
           B = build_matrix()
           @info "B = build_matrix()" pointer(A) pointer(B)
           A === B
       end
┌ Info: build_matrix: 
└ pointer(A) = Ptr{Float64} @0x00007fd15df7a080
┌ Info: A = build_matrix()
└ pointer(A) = Ptr{Float64} @0x00007fd15df7a080
┌ Info: build_matrix: 
└ pointer(A) = Ptr{Float64} @0x00007fd15df7b340
┌ Info: B = build_matrix()
│ pointer(A) = Ptr{Float64} @0x00007fd15df7b340
└ pointer(B) = Ptr{Float64} @0x00007fd15df7b340
true

```

This applies to `@testset` since the macro creates a local scope using `let`.

```julia-repl
julia> @macroexpand @testset begin
           function build_matrix()
               A = zeros(2, 2)
               A[1, 2] = A[2, 1] = 1
               return A
           end
           
           A = build_matrix()
           B = build_matrix()
           @test A === B
           println(A === B)
           
       end
quote
# ...
            let
                #= /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 =#
                begin
                    #= REPL[26]:2 =#
                    function build_matrix()
                        #= REPL[26]:2 =#
                        #= REPL[26]:3 =#
                        A = zeros(2, 2)
                        #= REPL[26]:4 =#
                        A[1, 2] = (A[2, 1] = 1)
                        #= REPL[26]:5 =#
                        return A
                    end
                    #= REPL[26]:8 =#
                    A = build_matrix()
                    #= REPL[26]:9 =#
                    B = build_matrix()
                    #= REPL[26]:10 =#

```

Conversely, if we define `build_matrix` in global scope, then we will get `false`.

```julia-repl
julia> function build_matrix()
           A = zeros(2, 2)
           A[1, 2] = A[2, 1] = 1
           return A
       end
build_matrix (generic function with 1 method)

julia> @testset begin
           A = build_matrix()
           B = build_matrix()
           @test A === B
       end
test set: Test Failed at REPL[31]:4
  Expression: A === B
   Evaluated: [0.0 1.0; 1.0 0.0] === [0.0 1.0; 1.0 0.0]

```

---

<div class="post-metadata">

**Author:** ![raminammour](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raminammour/32/13572_2.png) [@raminammour](https://discourse.julialang.org/u/raminammour)\
**Post date:** [January 17, 2024, 9:37pm UTC](https://discourse.julialang.org/t/different-result-of-between-runtest-and-repl/108924/3 "2024-01-17T21:37:53Z")

</div>

`Test` is a red herring. Something is fishy here, I think it is worth filing an issue:

```julia

julia> g()=begin
           function build_matrix()
               A = zeros(2, 2)
               A[1, 2] = A[2, 1] = 1
               return A
           end
           
           A = build_matrix()
           B = build_matrix()
            A === B
           
       end
g (generic function with 1 method)

julia> h()=begin 
           function build_matrix()
               A = zeros(2, 2)
               A[1, 2] = A[2, 1] = 1
               return A
           end
           
           C = build_matrix()
           B = build_matrix()
            C === B
           
       end
h (generic function with 1 method)
julia> g(),h()
(true, false)

julia> @code_warntype optimize=:false h()
MethodInstance for h()
  from h() @ Main REPL[38]:1
Arguments
  #self#::Core.Const(h)
Locals
  B::Matrix{Float64}
  C::Matrix{Float64}
  build_matrix::var"#build_matrix#14"
Body::Bool
1 ─ (build_matrix = %new(Main.:(var"#build_matrix#14")))
│ (C = (build_matrix)())
│ (B = (build_matrix)())
│ %4 = (C === B)::Bool
└── return %4

julia> @code_warntype optimize=:false g()
MethodInstance for g()
  from g() @ Main REPL[36]:1
Arguments
  #self#::Core.Const(g)
Locals
  B::Any
  A@_3::Core.Box
  build_matrix::var"#build_matrix#13"
  A@_5::Union{}
Body::Bool
1 ─ (A@_3 = Core.Box())
│ (build_matrix = %new(Main.:(var"#build_matrix#13"), A@_3))
│ %3 = (build_matrix)()::Any
│ Core.setfield!(A@_3, :contents, %3)
│ (B = (build_matrix)())
│ %6 = Core.isdefined(A@_3, :contents)::Bool
└── goto #3 if not %6
2 ─ goto #4
3 ─ Core.NewvarNode(:(A@_5))
└── A@_5
4 ┄ %11 = Core.getfield(A@_3, :contents)::Any
│ %12 = (%11 === B)::Bool
└── return %12

```

Just renaming `A` to `C` yields expected results (the output of the `@code_warntype` with the same name `A` looks utterly confusing and confused)! This looks like a bug, on 1.9.0 at least.

Cheers!

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [January 18, 2024, 9:17am UTC](https://discourse.julialang.org/t/different-result-of-between-runtest-and-repl/108924/4 "2024-01-18T09:17:46Z")

</div>

I also tested this on 1.8 and 1.10 and got the same results. It really seems that `function` does not introduce a proper local scope if inside a `let` and I don’t think that this can be intended.

Another “funny” consequence of this is that `===` appears non-commutative:

```julia
let
    function test()
        A = zeros(1)
        return A
    end
    A = test()
    A === test(), test() === A # gives (false, true)
end

```

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [January 18, 2024, 9:28am UTC](https://discourse.julialang.org/t/different-result-of-between-runtest-and-repl/108924/5 "2024-01-18T09:28:14Z")

</div>

Actually I changed my mind and now think this is intended. From [the manual on scopes](https://docs.julialang.org/en/v1/manual/variables-and-scoping/#local-scope) (emphasis mine):

> A new local scope is introduced by most code blocks (see above [table](https://docs.julialang.org/en/v1/manual/variables-and-scoping/#man-scope-table) for a complete list). If such a block is syntactically nested inside of another local scope, the scope it creates is nested inside of all the local scopes that it appears within, which are all ultimately nested inside of the global scope of the module in which the code is evaluated. \*\*Variables in outer scopes are visible from any scope they contain […] unless there is a local variable with the same name that “shadows” the outer variable of the same name. **This is true even if the outer local is declared after (in the sense of textually below) an inner block**. When we say that a variable “exists” in a given scope, this means that a variable by that name exists in any of the scopes that the current scope is nested inside of, including the current one.  
> […]  
> When `x = <value>` occurs in a local scope, Julia applies the following rules to decide what the expression means based on where the assignment expression occurs and what `x` already refers to at that location:
> 
> 1. **Existing local:** If `x` is _already a local variable_, then the existing local `x` is assigned;
> 2. **Hard scope:** If `x` is _not already a local variable_ and assignment occurs inside of any hard scope construct (i.e. within a `let` block, function or macro body, comprehension, or generator), a new local named `x` is created in the scope of the assignment;

I don’t like all consequences of this (like the example in this thread) but this is the way it is…

Edit: So the correct way to write snippets above is with using `local` inside the inner function:

> [@mkitti](#):
>
> ```julia-repl
> julia> let
> function build_matrix()
> local A = zeros(2,2)
> A[1,2] = A[2,1] = 1
> return A
> end
> A = build_matrix()
> B = build_matrix()
> A === B
> end
> false
> 
> ```

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [January 18, 2024, 1:49pm UTC](https://discourse.julialang.org/t/different-result-of-between-runtest-and-repl/108924/6 "2024-01-18T13:49:10Z")

</div>

`@testset`’s local scoping is [documented](https://docs.julialang.org/en/v1/stdlib/Test/#Working-with-Test-Sets), though it doesn’t spell out how it’s different from the global scope like the scoping docs or this case.

> [@abraemer](#):
>
> I don’t like all consequences of this

The other more sinister possibility is that a local `build_matrix` can be reassigned, even by the function’s own call. The upside to local variables being reassigned by default within local scopes is this is how closures can work. Closures that reassign variables aren’t performant for now and are imo a bit strange, but there are situations where it’s called for. As long as you know the scoping rules and the `@macroexpand` isn’t too complex to check, closures won’t be much of a stumbling block, especially since in practice, you’d test globally scoped functions which can persist outside the test.

Something that can help is more descriptive names that differ inside _and_ outside `function` blocks. Defaulting to quick but meaningless names is not too good a habit even if scoping rules help you out.

---

<div class="post-metadata">

**Author:** ![raminammour](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raminammour/32/13572_2.png) [@raminammour](https://discourse.julialang.org/u/raminammour)\
**Post date:** [January 18, 2024, 3:37pm UTC](https://discourse.julialang.org/t/different-result-of-between-runtest-and-repl/108924/7 "2024-01-18T15:37:21Z")

</div>

After some thought, it does look like this is intended, though unintuitive. So either:

- Don’t use the same name of variables so that none of the “shadowing” behavior happens
- If a function is meant to be defined in an outer scope like `build_matrix`, then just define it in the outer scope (as @mkitti did)

I guess this is what allows, sometimes desirable, behavior like:

```julia
julia> f1() = begin
       function bla()
       return a+1
       end

       a=3
       return a,bla()
       end
f1 (generic function with 1 method)

julia> f1()
(3, 4)

```

whereby defining `bla()` an outer scope would be an error.

Hope this helps…