# New scope solution

**URL:** <https://discourse.julialang.org/t/new-scope-solution/16707>\
**Category:** Internals & Design\
**Tags:** proposal\
**Created:** [October 23, 2018, 8:58pm UTC](https://discourse.julialang.org/t/new-scope-solution/16707 "2018-10-23T20:58:16Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [October 24, 2018, 3:34am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/22 "2018-10-24T03:34:04Z")

</div>

If this soft global scope hack means that I won’t be able to copy some code from a test file or an examples file and paste it into the REPL or IJulia to reproduce the behavior, and vice versa, then I find this a very weird divergence of language semantics. I actually prefer the current behavior over such divergence, if votes count, and I prefer the 0.6 behavior over both.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [October 24, 2018, 3:43am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/23 "2018-10-24T03:43:57Z")

</div>

I am not sure if you know it but currently you could not copy from function body to global scope in REPL and expect same behavior.

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [October 24, 2018, 3:47am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/24 "2018-10-24T03:47:00Z")

</div>

I did not mean to copy from a function body, but to/from global scope in a test file.

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [October 24, 2018, 4:48am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/25 "2018-10-24T04:48:29Z")

</div>

It may avoid some confusion if the compiler emits a warning if the following two conditions are met:

1. A local variable is assigned, but never read before it falls out of scope.
2. A global variable with the same name exists.

Condition 1 would be checked statically when the block is compiled, and condition 2 would be checked at run-time.

Consider the example:

```julia
for x in 1:5
    if x == 5
        found = true
    end
end

```

Here, the compiler would see `found` as a local variable that is written but not read, and emits code that checks for the existence of a global variable and if so, gives a warning: “Did you mean `global found = true`?”

(If the global `found` is a `Bool` then the warning could suggest `found |= true` as an alternative.)

There’s still the issue that a `@show found` that the user puts in for debugging purposes would make the warning go away. I don’t think there’s a way to avoid this.

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [October 24, 2018, 5:00am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/26 "2018-10-24T05:00:00Z")

</div>

On a related note: How about allowing postfix `if` following an assignment, and making

```julia
found = true if x == 5

```

equivalent to

```julia
found = if x == 5
   true
else
   found
end

```

That would create a code-path that reads before writing, which is what we want in these cases.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 24, 2018, 6:11am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/27 "2018-10-24T06:11:59Z")

</div>

I appreciate the effort to find a solution to a difficult problem — thanks for thinking about this!

However, I am not sure I like the fact that to reason about the `global`/`local` status of `x`, I have to look at surrounding code, and all the branches. IMO doing this kind of reasoning is not something that humans are particularly good at (as opposed to compilers).

I find it problematic that if I comment out some lines (which I do sometimes for debugging or WIP code), `x` could flip back and forth between local and global. While I recognize that it can be a pedagogical challenge in some contexts, I find the _status quo_ of `v1.0` easier to reason about.

---

<div class="post-metadata">

**Author:** ![00vareladavid](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/00vareladavid/32/23521_2.png) [@00vareladavid](https://discourse.julialang.org/u/00vareladavid)\
**Post date:** [October 24, 2018, 7:20am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/28 "2018-10-24T07:20:03Z")

</div>

It seems the options are:

- local scope + error messages so beginners at least know whats going on
- local scope + SoftGlobalScope (or some other tooling) by default in the REPL
- global scope
- global scope + `local let` (or equivalent) to easily create locally scoped blocks
- DWIM scope
- ~~abandon Julia and go back to Python~~

A subset of users/devs will be unhappy with the final decision 🤷‍♂️

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [October 24, 2018, 7:36am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/29 "2018-10-24T07:36:10Z")

</div>

> [@00vareladavid](#):
>
> - local scope + SoftGlobalScope (or some other tooling) by default in the REPL

I’d also want better error messages in this case (in files).

This would also be my personal preference. It’s solid, non-breaking, and we don’t have to change (again) all stack overflow and discourse post answers related to this scoping issue. It would just subtly improve the situation.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [October 24, 2018, 7:53am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/30 "2018-10-24T07:53:26Z")

</div>

Sometime you want to break from nested for. What about this? This is current behavior:

```julia
julia> broken = false
       for i in 1:2
            # broken = false
            for j in 1:2
                if true broken = true;break;end
            end
            println("inner $broken")
            if broken break; end
        end
        println(broken)
inner false
inner false
false

```

What would we expect here?

> [@StefanKarpinski](#):
>
> Another possible variation is that if every path is one of the following:
> 
> - read-before-write (including read-only)
> - write-only
> - no use

We could look at “write only”/“no use” case:

```julia
found = false
for i in 1:2 
    found = true
    @debug_test found == true # it is used (in read mode) here if macro is expanded!
end

```

Debug version will be different from no-debug version!!

---

<div class="post-metadata">

**Author:** ![gdkrmr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdkrmr/32/2791_2.png) [@gdkrmr](https://discourse.julialang.org/u/gdkrmr)\
**Post date:** [October 24, 2018, 7:58am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/31 "2018-10-24T07:58:47Z")

</div>

I think rules should be simple and easy, this one is not! If you really feel that you need this rule, remove it for Julia 2.0 or before if possible.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [October 24, 2018, 8:19am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/32 "2018-10-24T08:19:04Z")

</div>

It is quite a mess in current behavior too. Look at Schroedinger’s cat:

```julia
julia> dead = false
       for j in 1:1
           if rand()>0.5 dead = true;end # cat is unlucky :(
           print("is shroedinger's cat dead? $dead")
       end
is shroedinger's cat dead? true

```

If we avoid dead cat is happy:

```julia
julia> dead = false
       for j in 1:1
           # if rand()>0.5 dead = true;end
           print("is shroedinger's cat dead? $dead") # cat is lucky! :) 
       end
is shroedinger's cat dead? false

```

But in case of cat’s luckiness experiment is broken:

```julia
julia> dead = false
       for j in 1:1
           if rand()>0.5 dead = true;end   
           print("is shroedinger's cat dead? $dead") # coder is not lucky :( 
       end
ERROR: UndefVarError: dead not defined

```

It seems that assignement (which not happened!) made `dead` variable local and undefined.

EDIT:  
Could this be optimized out in future?

```julia
dead = false
for j in 1:1
  if VERSION<v"1.0" dead = true;end # I want to check conditional programming here
  print("is shroedinger's cat dead? $dead")
end
ERROR: UndefVarError: dead not defined

```

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [October 24, 2018, 8:36am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/33 "2018-10-24T08:36:02Z")

</div>

I don’t think many people want to leave things as they are.

Both in SoftGlobalScope and in a function (any local scope) your example works as expected. It would also work if everything would default to global. So this is most likely going to be fixed, (almost) independent of what change will be made.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [October 24, 2018, 8:48am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/34 "2018-10-24T08:48:05Z")

</div>

> [@carstenbauer](#):
>
> So this is most likely going to be fixed, (almost) independent of what change will be made.

How it could be if one proposal want to check context and context is quite questionable as could be seen from my tests too?

What does really mean that variable is not used? Or is used “write only”?

Maybe I am wrong, could you explain it more please?

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [October 24, 2018, 10:04am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/35 "2018-10-24T10:04:54Z")

</div>

Error messages: [https://discourse.julialang.org/t/improving-error-messages-for-the-scoping-problem/16209](https://discourse.julialang.org/t/improving-error-messages-for-the-scoping-problem/16209).

Remember, the local/global decision for variables does not happen at runtime, it happens at compile-time (I think early in lowering?). I think Stefan’s solution is to follow unconditional `@goto` and always follow both branches (even for literal `if false`). So it would fix the following example:

```julia
julia> dead=true;
julia> let 
           @show dead
           @goto skip
           dead = false
           @label skip
       end
ERROR: UndefVarError: dead not defined

```

but not

```julia
julia> dead=true;
julia> let 
           @show dead
           if true @goto skip end
           dead = false
           @label skip
       end
ERROR: UndefVarError: dead not defined

```

The rule would be: Follow all pathes (without evaluating known conditionals). If there exists a write before read path, then the variable defaults to local. Otherwise, it defaults to global.

As a side note: The `while` gets evaluated in the outer scope, not the inner scope. That is probably confusing for some people as well:

```julia
julia> m=4; n=2; i=1; while i>0
       i = n
       @show i, n
       global m -= 1
       global n -= 1
       @show m,n
       m>0 || break
       end; @show m, n, i
(i, n) = (2, 2)
(m, n) = (3, 1)
(i, n) = (1, 1)
(m, n) = (2, 0)
(i, n) = (0, 0)
(m, n) = (1, -1)
(i, n) = (-1, -1)
(m, n) = (0, -2)
(m, n, i) = (0, -2, 1)

```

So, regardless of this scoping, a minimally invasive (very non-optimizing) `@code_semilowered` that produces valid julia source code with only `let` blocks and `@goto` would be nice for that. It would also teach people about the iterator interface.

---

<div class="post-metadata">

**Author:** ![jballanc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jballanc/32/328_2.png) [@jballanc](https://discourse.julialang.org/u/jballanc)\
**Post date:** [October 24, 2018, 10:08am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/36 "2018-10-24T10:08:35Z")

</div>

This proposed solution bears a striking resemblance to [escape analysis](https://en.wikipedia.org/wiki/Escape_analysis), which is a tricky beast but also key to some seriously powerful compiler optimizations. It is also, notoriously, the one optimization that Java can still not perform (well). I bring this up because I think it is worth considering the fix to this “bug” in the larger context of escape analysis.

Java has problems with escape analysis not only because it is a difficult optimization to perform, but also because the language was not designed with it in mind. With Julia, we have the opportunity to evolve the language in a way that would facilitate escape analysis.

I think the crux of the scope “bug” is the desire to create strongly bounded scopes. We want this because it simplifies escape analysis. If we state that any variable created within a `for` loop, or within a function, falls out of scope at the conclusion of the loop or function body, unless returned, then we only need follow the path of explicit returns to perform escape analysis. However, if some value within one of these scopes is assigned to a global variable then we must consider multiple escape routes. Consider, for example:

```julia
b = []
function foo()
  global b
  for i = 1:10  
    append!(b, i)
  end
end
foo()

```

There are, in this function, 10 values that have escaped the function scope. Still, because we must specify `global b`, analysis is relatively straightforward. The more complicated the rules become for determining when a variable might escape a scope, the more difficult it becomes to perform escape analysis.

The REPL throws a monkey-wrench into all of this, as it is essentially a never-ending function call. Nothing can escape the REPL, so we would like to relax some of the constraints around escape analysis in the name of “user experience”. The problem, of course, is that the REPL is not a function call.

* * *

In short, I am not in favor of this proposed solution because of how it potentially complicates escape analysis. I do think, however, that it highlights one potential path toward a more general solution. What if, instead of tweaking the rules for how variables might, or might not, escape from an inner scope, we allowed for outer scopes to explicitly _opt out_ of variable escaping? In other words, what if you could do the following:

```julia
module Foo
  locally_scoped() # => this call alters the scoping rules of the module
  b = []

  function bar()
    for i = 1:10 
      append!(b, i)
    end
  end

  function baz()
    @show b
  end
end

Foo.bar();
Foo.baz() # => 10-element Array{Any,1}: 1, 2, ...

```

This way, the REPL could evaluate in a module context wherein every variable is considered locally scoped, but we can still preserve the ability to perform escape analysis (in every other module).

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [October 24, 2018, 10:25am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/37 "2018-10-24T10:25:06Z")

</div>

Thanks for reaction! 🙂

> [@foobar\_lv2](#):
>
> Remember, the local/global decision for variables does not happen at runtime, it happens at compile-time (I think early in lowering?).

You are more experienced, could you tell me if there is way to make [conditional compilation](https://en.wikipedia.org/wiki/Conditional_compilation) similar to C++'s [#ifdef](https://en.wikipedia.org/wiki/C_preprocessor#Conditional_compilation)?

Could be @assert [optimized out](https://github.com/JuliaLang/julia/issues/10614) in future version if there are so subtle implication to variable scope?

> [@foobar\_lv2](#):
>
> As a side note: The `while` gets evaluated in the outer scope, not the inner scope. That is probably confusing for some people as well

Is it true? I am really confused as well! 😛

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [October 24, 2018, 10:29am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/38 "2018-10-24T10:29:24Z")

</div>

> [@Tamas\_Papp](#):
>
> I find it problematic that if I comment out some lines (which I do sometimes for debugging or WIP code), `x` could flip back and forth between local and global. While I recognize that it can be a pedagogical challenge in some contexts, I find the _status quo_ of `v1.0` easier to reason about.

Maybe there can be a balance. The “if read before write then the user refers to the global variable” is probably safe enough (if you were writing that, maybe while debugging code, you’d be getting an error so you are really not losing much). In case this could still cause confusion, I imagine there is always the option to allow this but throw a warning (`Read before write variable in a scoped block defaults to global: to avoid this warning add the keyword global`). The new user can decide to ignore the warning (or learn from it) and the advanced user can copy paste the for loop from function body to REPL anyway as in this scenario the warning doesn’t matter so much (and add `global` in production code). The warning also has the advantage that the user will suspect that fancier tricks, like:

```julia
myvar = 0
for i = 1:10 
  myvar = i
  i == 5 && break
end

```

may require the keyword `global` to work as intended.

OTOH the “if we write on the variable but never read, then it is local” is IMO a bit extreme and here I completely agree that it risks getting too confusing (some `@show` statements during debugging could cause things to flip). I’m also afraid that this change is technically breaking. That is to say, if some users wrote:

```julia
myvar = 0
for i = 1:10 
    myvar = i
end
@assert myvar == 0

```

His / her code would break. I imagine nobody would write something like this on purpose, but I wonder whether semver allows this kind of changes in a minor release.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [October 24, 2018, 10:36am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/39 "2018-10-24T10:36:35Z")

</div>

> [@jballanc](#):
>
> ```julia
> module Foo 
> locally_scoped() # => this call alters the scoping rules of the module
> 
> ```

we showed [above](https://discourse.julialang.org/t/new-scope-solution/16707/35) that scope definition of variables is decided in compile time before calling (it could be different in REPL though) doesn’t apply it here?

---

<div class="post-metadata">

**Author:** ![jballanc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jballanc/32/328_2.png) [@jballanc](https://discourse.julialang.org/u/jballanc)\
**Post date:** [October 24, 2018, 10:47am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/40 "2018-10-24T10:47:01Z")

</div>

Yeah, I just sketched this up quickly, and you’re right that it would likely have to be some sort of new keyword or compiler directive. Maybe:

```julia
locally_scoped module Foo
# ...
end

```

But the idea is that, semantically, this would be the same as magical macro that appended `global` before every variable definition.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [October 24, 2018, 11:00am UTC](https://discourse.julialang.org/t/new-scope-solution/16707/41 "2018-10-24T11:00:55Z")

</div>

> [@piever](#):
>
> > I find it problematic that if I comment out some lines (which I do sometimes for debugging or WIP code), `x` could flip back and forth between local and global. While I recognize that it can be a pedagogical challenge in some contexts, I find the _status quo_ of `v1.0` easier to reason about.
> 
> Maybe there can be a balance. The “if read before write then the user refers to the global variable” is probably safe enough (if you were writing that, maybe while debugging code, you’d be getting an error so you are really not losing much).

This is safe in static code. But it will be unsafe to add simple “read” line into code under this “solution”.

But I suppose you know:

> [@piever](#):
>
> (some `@show` statements during debugging could cause things to flip)

[Previous page](https://discourse.julialang.org/t/new-scope-solution/16707.md?page=1)

[Next page](https://discourse.julialang.org/t/new-scope-solution/16707.md?page=3)