# The manual's section on variable scope sucks
**URL:** https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522
**Category:** General Usage
**Tags:** documentation, scope
**Created:** [September 17, 2024, 8:03pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522 "2024-09-17T20:03:43Z")
**Posts on this page:** 20
**Page:** 2
### Author:  [@sadish-d](https://discourse.julialang.org/u/sadish-d)
#### Post date: [September 20, 2024, 2:02pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/21 "2024-09-20T14:02:43Z")
If there are n variables and m blocks, there are n + m of something scope-like. I just wish for language that distinguishes between them so we can talk about all n + m things. Maybe it’s variable-scope and block-scope.
---
### Author:  [@sadish-d](https://discourse.julialang.org/u/sadish-d)
#### Post date: [September 20, 2024, 2:07pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/22 "2024-09-20T14:07:55Z")
I didn’t mention the proposal of `:=` because it’s not happening until Julia 2.0. I wanted to see what we can do with what we have at hand, so documentation becomes important. I do agree that it would allow us to distinguish between declaration ( `let x`, `local x`, `f(x) = ...`), definition `:=`, and assignment `=`, which is at the source of a lot of confusion. But we should be prepared to see code that looks like `f(x:=0) := nothing` if we are going to use `:=` consistently.
---
### Author:  [@jar1](https://discourse.julialang.org/u/jar1)
#### Post date: [September 20, 2024, 7:07pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/23 "2024-09-20T19:07:21Z")
The difference is that `:=` would be for all reassignments whereas `outer` is for only reassignment into other scopes. Either one would solve the main problem. Python does the `outer` thing (“`nonlocal`”).
* * *
`begin end` is so verbose that I wouldn’t want to use it – personally I think `begin end` should’ve been `{ }` but nobody wants to hear that.
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [September 20, 2024, 11:05pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/24 "2024-09-20T23:05:13Z")
> [@jar1](#):
>
> The difference is that `:=` would be for all reassignments whereas `outer` is for only reassignment into other scopes.
I misread it then. So it’s `=` for the original definition and `:=` optionally(?) for following reassignments, which doesn’t directly choose the `outer` versus `local` scope but would naturally fit because it’s more obvious a `:=` versus `=` in a nested scope is `outer` versus `local`. Its static checks could involve more than a scope boundary.
On the other hand, I had imagined that explicit `outer`/`local` could evade the difference in default behavior between local and global scopes, allowing source code to behave the same between them, especially from macros intended to work in both. It didn’t really go anywhere because it was hard to imagine how metaprogramming could replace the default behavior with `outer`/`local` automatically when macro calls were involved.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [September 21, 2024, 6:45am UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/25 "2024-09-21T06:45:06Z")
I am wondering how diffflicult would it be to write a utility package that takes a piece of code, parses it, the spits out an annotated version that shows then scope of each variable.
---
### Author:  [@Eben60](https://discourse.julialang.org/u/Eben60)
#### Post date: [September 21, 2024, 10:08am UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/26 "2024-09-21T10:08:47Z")
Given the excellent outline by Tamas, that could be something what an AI assistant can probably produce. A nice case for a competition between different AI assistants?
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [September 21, 2024, 10:59am UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/27 "2024-09-21T10:59:02Z")
I am thinking about doing the PR myself if no one else is interested.
---
### Author:  [@sadish-d](https://discourse.julialang.org/u/sadish-d)
#### Post date: [September 21, 2024, 1:42pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/28 "2024-09-21T13:42:43Z")
You mean a PR for the manual or a package? The manual I can help with. I don’t think I am skilled enough to work on such a package, but I can help with testing.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [September 21, 2024, 3:24pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/29 "2024-09-21T15:24:58Z")
The language manual, of course.
---
### Author:  [@jar1](https://discourse.julialang.org/u/jar1)
#### Post date: [September 21, 2024, 5:58pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/30 "2024-09-21T17:58:01Z")
I think [GitHub - c42f/JuliaLowering.jl: Julia code lowering with precise provenance](https://github.com/c42f/JuliaLowering.jl) will be the right tool for this but it’s not ready yet.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [September 23, 2024, 11:14am UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/31 "2024-09-23T11:14:47Z")
Working on a PR already, but I am wondering about the best way to include explanations within the code. Instead of just text explaining each code snippet, I think it would be great if examples also came with simple explanations as comments. (This was the practice for a lot of the discussions).
Currently I am leaning towards Unicode box characters, as in
```julia
function f(x)
# scope of let
# ╱ scope of function f
# ╱ ╱
let x = x + 1
# scope of let
#╱
x += 2
end
x
end
```
The reason for `╱` is that even if the variable is the first character in line, it can come after the `#` and still be well-aligned. Suggestions/comments are appreciated.
---
### Author:  [@sadish-d](https://discourse.julialang.org/u/sadish-d)
#### Post date: [September 23, 2024, 1:33pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/32 "2024-09-23T13:33:50Z")
I find it easier to read when the `#` are all in the leftmost column:
```julia
function f(x)
# scope of let
# ╱ scope of function f
# ╱ ╱
let x = x + 1
# scope of let
# ╱
x += 2
end
x
end
```
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [October 16, 2024, 6:02pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/33 "2024-10-16T18:02:01Z")
_What follows is my draft on how I would organize a chapter, without the examples. Comments are welcome, especially if I got something wrong._
## Preamble
This is all about _lexical scope_, ie the location of definitions matter, not the nesting of the stack.
## Some terminology, and the easy part
A key concept is _scope blocks_, introduced by
1. `module`, `baremodule`
2. `struct`, `for`, `while`, `try`,
3. `macro`, `function`, `let`, comprehensions, generators
4. `if`, `begin` (yes, more on this later)
The easy part: what a variable _refers to_ is determined like this. You keep checking the next outer block, including modules, until you find the variable in the scope of the block (implicitly, what you type in the REPL is in `Main`), or error. Notably,
1. this does not cross submodule barriers, when you reach the innermost module (which is called the “toplevel”) you are done,
2. order of evaluation does not matter within the scope block, if a variable belongs to a scope block, it does so for the entire block (but may not be assigned, just declared — this is about scope, not values).
In this, Julia is like all languages descended from Algol 60. Nothing to see here.
For historical reasons, variables at the toplevel are called _global_, while the rest are called _local_. Even though all of them are within blocks, either implicitly or explictly. Deal with it.
## The tricky part
The tricky part: how variables are _introduced to_ a scope blocks.
### Directly
Some blocks just name variables, which are introduced into the scope of the block. Notably,
Each `function` argument is in the scope for subsequent arguments and the function body:
```julia
function f(x,
y = x; # x is in scope
z = y) # x and y are in scope
# x, y, z are in scope
end
```
and the same holds for `do` blocks, `macro`, and, needless to say, short forms like `f(x) = ...`.
Comprehensions and generators:
```julia
[scope_of_x for x in ...]
(scope_of_x for x in ...)
```
`for` blocks for variables immediately following the keyword:
```julia
for x in ...
# scope of x
end
```
And finally `let`, the explicit purpose of which is to introduce a block with scope for a list of variables, each of which is in scope for the subsequent ones:
```julia
let x = 1, y = x
# scope of x and y
end
```
Note that you separate with commas, and it only affects variables before the line break.
### By assignment
If you assign (`x = ...`) _within_ a scoping block, the following happens:
1. For `function`, `do` blocks, the body of comprehensions & generators, and `macro`, a new variable is introduced. This is “hard local scope”.
2. For `for`, `while`, `try`, and `struct`, the rules of “soft local scope” apply. Specifically,
3. For `module` and `baremodule`, assignment creates a global variable at the toplevel (this covers `const` and function declaration, too).
4. For `if` and `begin`, the scope will be the innermost enclosing block that is not `if` or `begin`.
Note that destructuring may count as assignment (eg when within a function):
```julia
(; x, y) = ...
```
or directly introduce variables:
```julia
function f((x, y),)
...
end
```
In any case, the rules above apply.
### By declaration
`global x` will declare the variable in the _enclosing module at the toplevel_, no matter where you use it. And you can use it everywhere.
`local x` will declare the variable to have scope in the enclosing block. It is rarely used for `function` and friends, because that is the default for hard local scope. But you can use it to override the behavior above for soft local scope. You cannot use it in `struct`, `macro`, `module`, `baremodule`, but you can use it for `if` and `begin` _at the toplevel_.
Note that either of these constructs followed by assignment (eg `global x = 3`) is a shorthand for declaring a variable with the given scope, the assigning to that variable.
---
### Author:  [@albheim](https://discourse.julialang.org/u/albheim)
#### Post date: [October 16, 2024, 8:09pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/34 "2024-10-16T20:09:10Z")
> [@Tamas\_Papp](#):
>
> this does not cross submodule barriers, when you reach the innermost module (which is called the “toplevel”) you are done,
Wouldn’t this be the outermost?
> [@Tamas\_Papp](#):
>
> keep checking the next outer block
---
### Author:  [@abraemer](https://discourse.julialang.org/u/abraemer)
#### Post date: [October 16, 2024, 8:23pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/35 "2024-10-16T20:23:53Z")
Great draft 🙂
Here are some (hopefully helpful) comments:
> [@Tamas\_Papp](#):
>
> If you assign (`x = ...`) _within_ a scoping block, the following happens:
>
> 1. For `function`, `do` blocks, the body of comprehensions & generators, and `macro`, a new variable is introduced. This is “hard local scope”.
I believe this is wrong or at least worded in a way that reads wrong to me. Considering your description I would expect this function to print `1` but it prints `5`
```julia
function test()
x = 1
map(1:5) do y
x = y
end
println(x)
end
```
Without the surrounding `function` block you would be correct. The rules you describe here are valid only when the enclosing scope is a `global` scope.
Perhaps I didn’t understand what precisly you mean by “within a scoping block”.
EDIT: Thinking some more about this: Did you mean the argument names are always local to the function body shadowing outer bindings even if local? That is correct indeed, i.e. this prints `1`
```julia
function test()
x = 1
map(1:5) do x # changed this to shadow the function's x
end
println(x)
end
```
I’ll leave in the rest below anyways.
My mental model is the following (might be inaccurate):
Julia has 2 types of scopes: global and local. Global scope is basically synonymous with “toplevel”. Local scopes can be inside each other with a single global scope on the outermost level.
The difference in behavior between “soft” and “hard” local scopes for variable assignment depends solely on the enclosing scope the name is found in:
- If the enclosing scope is local there is no difference. No new binding is created.
- If the variable name is found at toplevel/global scope, then a “soft” scope reuses that binding, while a “hard” local scope will introduce a new binding.
So this section about assignments might need some corrections/rephrasing for clarity.
Not an error but earlier
> [@Tamas\_Papp](#):
>
> - order of evaluation does not matter within the scope block, if a variable belongs to a scope block, it does so for the entire block (but may not be assigned, just declared — this is about scope, not values).
you talk about that the “variable belongs to […] the entire scope block”. This is true but feels very unintuitive to me (perhaps just because it did a lot of Python before Julia). So would accompany this with an example that showcases this. We can reuse my example function from above:
```julia
function test_thisworks()
map(1:5) do y
x = y
end
println(x)
x = 1 # assingment means x belongs to function scope
end
function test_thiserrors()
map(1:5) do y
x = y
end
println(x)
#### x = 1 # no assignment, x is only local to closure
end
```
---
### Author:  [@Nathan\_Boyer](https://discourse.julialang.org/u/Nathan_Boyer)
#### Post date: [October 16, 2024, 9:33pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/36 "2024-10-16T21:33:09Z")
I applaud you tackling this and appreciate your attempt to categorize and group the behavior better. Now some critques.
* * *
In general, I think you’ve brought the weedy details up earlier than indicated in your first post. For example, instead of branching if’s of text here. I would rather the documentation just explain the simple, 80%-of-the-time case up front and then throw on some asterisks which link further down. The linked sections can then explain each exception and why it is necessary.
> [@Tamas\_Papp](#):
>
> For `for`, `while`, `try`, and `struct`, the rules of “soft local scope” apply. Specifically,
>
> 1. If there is no variable with the same name at the toplevel (a “global”), a new variable is introduced _locally_, with scope within this block.
> 2. If there is such a variable at the toplevel,
> 1. in interactive contexts (notebook, REPL) that variable is assigned to and _no local variable is created_, while
> 2. in non-interactive contexts a local variable with scope within this block is created.
I am the author of the two admonitions currently at the top of the scope documentation which I added to answer my questions:
1. Isn’t there a simple rule to explain all this if I don’t do anything weird?
2. Why all this technical detail? Just tell me why my loop isn’t working.
* * *
To the last point, I’m not sure any of your current text explains the nuance required for while loops. You only really get an error if you try to access and declare. If you do only one or the other, it will run.
```julia
i = 0
while i<5
# alone, runs and prints 0
println(i)
# alone, runs and prints 10
i = 10
println(i)
# both together error.
end
```
* * *
> [@Tamas\_Papp](#):
>
> For `for`, `while`, `try`, and `struct`, the rules of “soft local scope” apply. Specifically,
>
> 1. If there is no variable with the same name at the toplevel (a “global”), a new variable is introduced _locally_, with scope within this block.
I don’t think this is true because the below errors with nothing in global scope. It tries to re-use the outer (non-global) x.
```julia
let
x::String = "hello"
for i in 1:1
x = 3
println(x)
end
end
```
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 16, 2024, 10:50pm UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/38 "2024-10-16T22:50:26Z")
> [@Nathan\_Boyer](#):
>
> I don’t think this is true because the below errors with nothing in global scope. It tries to re-use the outer (non-global) x.
>
> ```julia
> let
> x::String = "hello"
> for i in 1:1
> x = 3
> println(x)
> end
> end
>
> ```
I think it just needs to be clarified that soft-scope is active at the global level. So it only applies to a global `for` loop or whatever (and `struct` should probably be dealt with separately).
Within an already existing local scope, soft-scoped constructs like `for`, `while`, `try`, etc. introduce no new scope. It’s only at the global level that they do this sorta-half-not-really-locally-scoped behaviour.
---
### Author:  [@sadish-d](https://discourse.julialang.org/u/sadish-d)
#### Post date: [October 17, 2024, 3:58am UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/39 "2024-10-17T03:58:33Z")
> [@Tamas\_Papp](#):
>
> what a variable _refers to_ is determined like this. You keep checking the next outer block, including modules, until you find the variable in the scope of the block
Is this actually how it the implementation works? Or is this just a mental model you’re proposing?
---
### Author:  [@sadish-d](https://discourse.julialang.org/u/sadish-d)
#### Post date: [October 17, 2024, 4:01am UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/40 "2024-10-17T04:01:15Z")
It would be nice if 80% of common confusions about scope were addressed in the beginning of the chapter. But to me, this is secondary to accuracy and completeness. And I don’t know if we can achieve accuracy and completeness without better language.
@Tamas_Papp, you’ve written:
> If you assign (x = …) within a scoping block… For function…a new variable is introduced.
One way to “unit test” the manual is to find counterexamples. So consider this counterexample to the above statement:
```julia
function f()
x = 0
x = 1
return nothing
end
```
The `x = 1` in the example does not create a new variable, it assigns to an existing one.
This ambiguity illustrates why I think we should distinguish between declaration and assignment from the very beginning. When you say variables get assigned “directly” (`f(x) = ...`), there is still declaration happening there, is there not? When you say `x = ...` is assignment, there might be declaration happening there too, as is the case with `x = 0` in the example above. And You haven’t mentioned `let` blocks in your “By assignment” section. Those can get tricky.
I’ve said this already, but I similarly think that we should not mention hard vs soft scope, at least not till the end of the chapter. The distinction is just a matter of the default behavior of the REPL. While I agree that many people probably run into this early on, emphasizing the distinction creates an unnecessary cognitive load.
If I see a piece of Julia code, I want to be able to pull up the manual next to it, and make sense of the code. Maybe that means the best way to explain variable scope is through a flowchart. And I might find it annoying to go through 15 branches of yes/no to understand a variable’s scope, but I would live with it if it gave me the right answer.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [October 17, 2024, 7:01am UTC](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522/41 "2024-10-17T07:01:44Z")
> [@albheim](#):
>
> Wouldn’t this be the outermost?
No, going from inside out, it is the first module (ie the narrowest submodule). Eg
```julia
julia> module Foo1
x = 1
module Foo2
x = 2
f() = x
end
end
Main.Foo1
julia> Foo1.Foo2.f()
2
```
> [@abraemer](#):
>
> Did you mean the argument names are always local to the function body shadowing outer bindings even if local?
Yes.
[Previous page](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522.md?page=1)
[Next page](https://discourse.julialang.org/t/the-manuals-section-on-variable-scope-sucks/119522.md?page=3)