Another possible solution to the global scope debacle

Internals & Design
163

The issue was discussed at length, see eg

164

I think the cure is getting worse than the disease. We went from a rule with complex subcases (soft scope, hard scope, etc) to a clean one which then some people found counterintuitive in some interactive contexts. Going back to special cases/constructs again may not be an improvement.

3 Likes
165

Is there a branch with this proposed behaviour,
I think I would really need to try it for a while to see how it feels.

Am I right is saying this would make more code act the same inside or outside a function?

1 Like
166

Frankly, I don’t think the discussion is about language design, but about how to accommodate instructors so that they don’t have to change their habits.

3 Likes
167

This is what I understand. The problem I had with this statement (given my examples at the beginning of this thread) is that beginners would start to think that all code would act the same which is not true. So essentially copy-pasting code from function body to global REPL is not safe anyway.

Other case of not the same behavior is unintended overwriting of globals vs. failure to redefine a constant (due to the issue related to whether a function was evaluated or not in a global scope).

In other words: although the meaning of code under the proposed rule can be determined statically it is not possible to say statically if it will work or fail because it depends on the state of global REPL.

This is what I mean:

julia> for i in 1:10
       global cos = 10
       end

julia> cos
10

julia> sin
sin (generic function with 12 methods)

julia> for i in 1:10
       global sin = 10
       end
ERROR: cannot assign variable Base.sin from module Main
3 Likes
168

Couldn’t those problems be solved almost independently from each other?
Namely the language by default will do what it right design wise yet it will also be flexible to allow building an environment (By loading package / One time defined macro) to satisfy demonstrators’ needs?

169

You would only opt in when you are explicit, so it’s fine for new people. And it would be using a keyword that already exists, for the same kind of concept. Actually, I think it could end up simpler to understand than the proposed solution. Since in the proposed solution, I think you could only use “local” inside scoping blocks that are in global scope (it wouldn’t work inside an if but yes inside a for?). With local blocks, one could make it that local vars are never allowed in global scope. If you want it, just wrap the block in local and make explicit globals if necessary.

This is how I would ~ see one option, with the local let thing. If I’m confused, someone correct me:

  1. Every bit of code is in either global or local scope. (like now)
  2. Outermost stuff is by default global scope (like now)
  3. Function contents (or blocks qualified with local) are in local scope
  4. When in global scope, every variable is global (like now)
  5. When in local scope, doing x = ... makes x a local var for the whole block. Unless it was somewhere qualified with global. (like now)
  6. Scoping blocks (for, let, …, not if or begins) in local scope, introduce a nested local scope. (like now)
  7. Nested local scopes inherit their parent local scopes vars, unless marked with local. The new vars that didn’t exist, x=... belong just to them. (like now)

With the current proposal as is, one would have to remove the parentheses’ contents from 3. and add

  1. Scoping blocks in global scope allow for marking variables with local

The current behaviour would require removing 3., and changing 6. to

  1. Scoping blocks (functions, for, let, …, not if or begins) always introduce a new (nested) local scope
170

I wonder what the purpose of a “non-local” let is. What would we lose if let always and everywhere introduced local scope to its contexts?

171

At least

  1. The more exceptions, the more complex it becomes
  2. According to Jeff, for works like a repeated let so, making them behave differently feels to him like more special cases.
  3. Making for default to local defeats the whole purpose of the proposal.

I wonder if 2. is only a core dev problem though, and as the viking said,

The new user can go without using let in global scope for a while I suppose.

172

Good point — if we maintain the equivalence of for with a transformation to let, then this is indeed the case.

The more I learn about the issue, the happier I am with the 1.0 choice (as is).

2 Likes
173

That’s right, a categorical error about the type of problem, which hopefully we won’t all have to pay for.

174

I believe that https://github.com/JuliaLang/julia/issues/28789#issuecomment-414799023 was or is an attempt to do that ?

Ah, here it is.

2 Likes
175

Ditto for me. I find the current behaviour simple and consistent. I’m also a bit concerned about let blocks behaving differently inside vs. outside functions under the new rules:

function f()
    let x = 1
        y = x + 1
    end
    y # undefined
end

# versus
let x = 1
    y = x + 1
end
y # 2
1 Like
176

I am one of these instructors who is arguing for a change in scoping behavior. For the record:

  • For my own personal use of Julia, I really like the 1.0 scoping rules. They’re simple and consistent, and it’s easy to add a global keyword for interactive snippets in the REPL. I’m still a bit on the fence, but I may even prefer the 1.0 system over the proposal in this thread (again, for my own use).
  • For my teaching, 1.0 scoping requires a bit of extra effort but is no great hardship. I can choose between taking a small detour to explain scoping up front, or I can punt and have the students use SoftGlobalScope.jl in the first sessions and explain why later.
  • My sole concern in pushing this is that I think 1.0 scoping is too unintuitive for unchaperoned beginners, and I think it will significantly hurt their inital impression of the language and may even slow adoption rate long term. Stefan also seems to be coming around to this view. I’m arguing for a change simply because I think my role as a teacher gives me a canary-in-a-coal-mine perspective that much more experienced Julia coders than I may lack.

Basically, please don’t kill the messenger misunderstand the message. We teachers are not griping about what we have to teach, we’re (or rather many of use are) worried about the confusion of our students.
[EDIT: clarified last two sentences, thanks Tamas.]

4 Likes
177

I may misunderstand this, but I don’t think the tone was hostile at any point.

Please keep in mind that many people here teach or taught programming to non-CS audiences, on both sides of this issue.

1 Like
178

I agree with your first two points. This last one is something which was suggested here to be solved by loading SoftGlobalScope.jl or similar by default. However, from what I understood from the discussion it is not clear to me if that would really solve everything or what pitfalls there might be with that approach and how this might interact with semantic versioning. I will await the result of the discussion and see what to do then.

179

Sorry, but I think this is not correct.

While the second let block returns the value 2 indeed,
asking for y still gives ERROR: UndefVarError: y not defined

180

First of all, thank you @jeff.bezanson (and the other core devs of course) for spending a lot of time and effort on this issue.

Concerning feedback on the proposed solutions, I like that:

  • It is very consistent and easy to explain. IIUC it says that: “things outside of functions are global unless marked as local explicitly”
  • It is IMO quite intuitive for users novel to programming (as opposed to the naive but quite common “I have defined x here, why does it say it’s undefined?”)
  • It only special cases functions, which is nice as they are arguably the most fundamental Julia construct

Things that could be added (already mentioned above):

  • A fix to for outer i = ... in top level scope, now it errors
  • local for ... and local let ... as in some cases the local default seems preferable (especially in “production code” or modules)

The second point is particularly relevant in a transition period as the warning would go from:

In the future in top-level for blocks variable assignment will default to global.
Annotate all assignment explicitly as local or global.

to

In the future in top-level for blocks variable assignment will default to global.
Use local for to retain the old behavior.

However it seems to me that this change is a bit drastic (especially as the proposed option hasn’t been tested yet) and I have the impression that is not necessarily an improvement in module code. For example, IIUC, with this proposal if I add a for loop at top level in a module, all the variables that I use will become global variables of the module and accessible from outside and I also risk to overwrite relevant globals that I am exporting.

With this in mind, I wonder whether the solution proposed in this github comment would be better: hard scope in modules (that are “production code”) and soft scope otherwise (scripts and REPL). The inconsistency may be annoying but by the time a user can write a module, I’m sure he/she can deal with this scope subtlety. In this scenario though what a “script” does when executed is a bit tricky to decide but one could opt in or out of this behavior by adding a line at the beginning of the file, say softscope(true) or something similar.

181

Concerning the wave of questions about this on discourse / slack / stack overflow etc., I wanted to point out that this could probably be reduced by a better error message. Right now one gets:

julia> x = 1
1

julia> for i = 1:10
       x = x + 1
       end
ERROR: UndefVarError: x not defined

Whereas something like:

x inside the for loop refers to a local variable, use global x to modify global variable x

would probably be better.

23 Likes
182

This is what I have suggested in my post. However in the meantime I think I have changed my mind and prefer the solution proposed in this thread:

  • no global annotation needed, IJulia could drop the ‘automatically-insert-global-hack’ (hopefully I’m not wrong here, didn’t use IJulia in 1.0 yet). REPL doesn’t ‘need’ a similar ‘~hack’.
  • no (?) other language has such scoping rules in the for loop (is this true?)
  • no surprises (1. there was a mention from a Matlab user at the breakfast table: you have to fix this – I think this is strong signal and bears weight, 2. there was a ‘bug report’ exactly about this problem – another signal of unexpectedness)
  • advanced users are more fit to declare what they want, i.e. write local when needed (local for is a cool idea, btw.)
  • copy/paste from REPL to functions works right away without ‘global adjustment’
  • former hard/soft scope complication is still solved

The only price is that the for scope is now global ([Edit]: maybe this is really bad, I’m not sure). – This said, I don’t feel competent enough to really state an opinion…

3 Likes