Jeff, you’re a brave man posting this here!
I too think the discussion has run its course. For the technical proposal, a Julep would be great. Whilst your initial description is nice and clear, for most of us (well, at least for me), who are not thinking about scope the whole time, a more elaborate description including examples of 0.6 vs 1.0 vs this proposal would be beneficial.
I think the issue is described as this. There are two competing things we want:
The change which introduced this “debacle” isn’t necessarily bad, it’s just gradient decent to solve (1). After that change, scoping rules were easy to explain. However, now it was quite simple to run into a case where when teaching first time students, you have to explain scoping rules (2).
Jeff’s proposed change is very good on both (1) and (2). Everything outside of a function is global, and so you won’t run into things not being defined outside of the loop.
It does make more variables global “by default”, but I’m not entirely sure that’s something that is a major issue. @testset and such tools are supposed to enclose tests into a local scope so that way globals don’t “leak out”, so I think that it’s safe. I wouldn’t worry about the performance aspects either since I think it’s safe to say “if you want performance, make it a function” and small amounts of locality in the REPL won’t do much to the grand scheme of things.
IMO this is a very positive way forward.
I value the ability to write code in IJulia for example, with as many variables as I feel like, knowing they won’t leak and without using local all over the place (or enumerating every variable on top of the let)
If I understand this correctly, this would be gone with the proposal. Is there some tweak that could fix it? What if we added let to the bag, together with functions, that default to local scope?
Not having a way to mark a piece of code to behave “as if in a function” without wrapping it and running it separately doesn’t feel ideal. One is forced keep track of all the globals at any point, that may be far away, so as not to choose those names and accidentally step over a global.
At this point, as you said, you are convinced there’s no ideal solution. But, we should have easy options to explicity have one behaviour or the other. This proposal, I think, leaves us with only one option.
Bless you for addressing the actual proposal.
That was our thinking as well.
Here are the examples. Julia ≤ 0.6 behavior:
@assert !isdefined(:x)
for _ = 1:1
x = "something"
# creates local `x` because there is no global called `x`
end
# global `x` is still undefined because the above `x` was local
@assert !isdefined(:x)
global x # or assign it a value
@assert !isdefined(:x) # `x` "exists" but is not defined
for _ = 1:1
x = "something"
# assigns to global `x` because it "exists"
end
@assert x == "something"
Julia 1.0 behavior:
@assert !@isdefined(x)
for _ = 1:1
x = "something"
# always creates a local `x`
end
@assert !@isdefined(x)
x = "something else"
for _ = 1:1
x = "something"
# even if a global `x` is defined
@assert x == "something"
end
@assert x == "something else"
for _ = 1:1
global x = "something"
# to assign to global `x` must use `global` keyword
end
@assert x == "something"
Proposed behavior:
@assert !@isdefined(x)
for _ = 1:1
x = "something"
# always assigns to global `x`
end
@assert x == "something"
for _ = 1:1
local x = "something else"
# to get a local `x` must use `local` keyword
end
@assert x == "something"
# `x` is still "something" because the inner `x` was local
In particular, the key is the meaning of this snippet of code:
for _ = 1:1
x = "something"
end
In 0.6 the meaning of this code depends on whether a global variable named x “exists” or not, which is not quite the same as if it is defined or not—it can exist without being defined (if it is defined, of course, then it must exist). If global x does not exist, then it creates local variable x inside of the loop. If global x does exist, then it assigns to that global x.
In 1.0 the meaning of that code does not depend in any way on what globals exist or are defined: it always defines a new x local variable inside the loop body.
In the proposal, the meaning of the code also does not depend on what globals exist or are defined but in the opposite way: x always assigns or updates the x global variable.
We didn’t really address let explicitly. There’s two ways to go with that:
Have the body of let behave like a loop body so assignments to variables that are not let-bound or declared local will create or update globals.
Have the body of let behave like a function body so assignments to variables that are not declared global will create or update locals.
I think I prefer (2) myself, but @jeff.bezanson had compelling reasons for (1) which I can’t recall at the moment. Perhaps he can refresh my memory.
Currently for loops behave like “repeated let” — they’re exactly like let blocks but with a backwards branch. So continuing to treat them the same feels like fewer special cases to me.
Maybe one could use local for, local let, local begin. In any case, if the “only functions” thing is not strictly part of the proposal, then I see no problem.
local for is kind of a clever idea; that had not occurred to me.
For 1.0, we wanted all forms with scopes to behave the same. Since an assignment inside a function simply must default to creating a local, that implies default-local for everything. So any other approach involves exceptions. I’d prefer to have as few exceptions as possible. I find it easier to say “except functions” than to have to list several things. Functions are also special in that they’re the only way to run code in an order other than top-to-bottom.
If I may I will bump my question: Will the proposed change handle globals uniformly? Meaning the same rules for the REPL as for modules?
Yes, that is one of the goals.
Excellent! And I wouldn’t worry too much about the grumbling concerning the versions and such. Julia quality speaks for itself, so just as long as the change is for the better, I think everyone will just eventually stop complaining and things will be fine.
Won’t this transition be very annoying? Every single variable outside of a function would require annotations (or at least issue a warning) until about a year from now?
Only when they are assigned inside a scope block. Still a lot of variables, but not every one.
@jeff.bezanson’s proposal indeed seems to me the optimal one if we need to address point (2) by @ChrisRackauckas of not needing to explain anything about scopes when showing this code to a newcomer
x = 0
for i in 1:2
x = i
end
x # EDIT: x == 0 in v1.0, but x == 2 with the proposed change
I don’t really like it, but given the assumption that we want to solve (2), it does seem the best course of action. But then I would ask, what if I show the newcomer this other code?
x = 0
f(i) = (x = i)
f(2)
x # EDIT: x == 0 in v1.0, and also x == 0 with the proposed change
They will still be puzzled that x is not equal to 2, no? And there is no way we should make function scopes leaky to address that!
Currently:
@local that does (()->$(esc(code))() if no alternative is provided but I’d really like one to be provided).Allow me to add one character to your first example and you get one of the most common constructs in programming. Something like this is almost guaranteed to come up within your first hour of playing with Julia, WAY sooner than you want to discuss the subtleties of scoping:
x = 0
for i in 1:2
x += i
end
x
As for the second example …
x = 0
f(i) = (x = i)
f(2)
x
Here the explanation is rather intuitive, even for students coming from a quasi-language like Matlab: “In Julia, variables in functions are local by default. Add the global keyword if you want to change a global variable.”
And that’s pretty much all you have to say about scope until you start putting closures inside other functions. Which is a much more appropriate time to have that discussion.
So what is your point exactly? If you find it acceptable to say to the student “In Julia, variables in functions are local by default”, why not just say “In Julia, variables in functions, for and let blocks are local by default"? That’s all it takes to understand v1.0, really, and it makes for a good lesson in coding hygiene…
Actually an error in 1.0, x == 3 with the proposed change and in <=v0.6.
It’s a cost benefit thing.
The more exceptions to the rule, the harder it is to grasp. Functions are an unavoidable exception.
Many more people that never read about scope can get started without bumping into surprises about function scope than about for/let.
Thanks, I just copied the whole examples without reading the comments. Fixed now. My argument still stands though.