Another possible solution to the global scope debacle


#215

For what it’s worth, one of the more confusing things is that the name resolution is within the scope and a syntactic, not semantic thing. By this, I mean:

julia> x=1;

julia> function f()
       @show x
       false && (x=1)
       nothing
       end
julia> function g()
       @show x
       #false && (x=1)
       nothing
       end
julia> f()
ERROR: UndefVarError: x not defined
julia> g()
x = 1

In principle, I would prefer a splitting into UndefGlobalVarError (no local binding of that name exists, and at runtime there is no global of that name) and UnInitializedVarError (a binding for this symbol exists in the scope, but has not yet been assigned at time of use – all control flows leading to a use before assignment must give us this error, but this may need to be tracked at runtime due to @goto, if the halting problem for the specific user code is too hard for the compiler).


#216

FWIW, I really like @jeff.bezanson’s proposal, along with introducing local for and global for (and let), which defaults to local for now, with the option of changing the default later (whether in 1.3 or 2.0) after some experience has been gained, as advocated by @Liso and @oxinabox. Apart from being quite clean, this strategy should also alleviate @tim.holy’s concern. Importantly, code will continue to work the same in scripts and at the REPL; changing that seems worse than the original issue to me.

For teaching, any discussion about scope can thus be postponed by just introducing the global for concept at first; perhaps this could be simplified further by introducing for! as syntactic sugar as proposed earlier (though that begs the question of what happens to naked for if/when the default is changed down the road) In addition, perhaps a warning could be issued whenever a local variable is introduced that shadows a global, in addition to a more helpful error message as proposed in #29585, silenceable by macro or environment variable. That might take much of the sting out of Example 1 of the original issue. For Example 2, it would technically even be non-breaking (as it currently errors) if total_lines were initialized to the value of the global in addition to the warning, making the example work as intended.

With these measures, we might even find that the flood of discourse and stackoverflow questions subsides to a point where changing the default may not even be warranted anymore, or could at least be postponed until 2.0, hence honoring semver.


#217

Thinking a bit more about this, for! i = ... could additionally be made to imply for outer i, and this could be how it’s explained in a teaching context, hence avoiding any reference to scope: just say it means that “variables can get overwritten by the loop”. If/when the default is changed to global for later, for! i will simply continue to imply outer. This might be simple enough for non-CS students to grasp. @stevengj?


#218

I am not sure why one would jump through hoops just to avoid mentioning scope (briefly, of course).

Maybe I was lucky to teach some exceptionally bright students, but usually even students without prior programming experience (who were nevertheless interested in programming, which is why they took the course) grokked scope rather quickly. After all, it is a very intuitive concept, and someone taking courses at or above a BA/BSc level in STEM or data-based social sciences must encounter abstractions that are much more difficult. Teaching even basic programming without scope is like teaching linear algebra without bases.

I rarely saw for loops that accumulate in global scope in the wild in Julia, even for v0.6, because it is bad style.

If it is not something we would do because it is not good practice, why would we teach it? Isn’t this a disservice to the students?

PS: Sorry @s-broda, of course not all of this is in reply to your comments. I just find the “teaching” argument not super-convincing.


#219

It’s perfectly good style for interactive exploration, where you usually don’t write functions. Not all coding is about performance or reusability.


#220

As far as I am concerned, the idea is not to avoid introducing scope, but to postpone the discussion to a more appropriate time than literally the first 20 minutes of the first lecture (which may not even be on programming; in my case it’s finance). It’s easy to forget how hard these concepts are for students from non-STEM backgrounds.

In any case, just from the number of people running into this issue, it is clear that there is an issue here, presumably not just for students but also people who may not enjoy the luxury of a carefully designed course. So I think it’s best to keep the thread focused on the merits of the proposed solutions.


#221

I am not so sure about this, since we don’t know the base (to be fair, the this can go both ways, since people could encounter the issue and not show up here or other forums).

Given the difficulties with the various solutions that are apparent in this thread, I think we should keep “not changing semantics, but giving helpful error messages” on the table as one of the solutions, at least in the short run. Strictly speaking of course you are right, since anything not discussing this particular solution is off-topic in this thread, but I feel that has happened already.


#222

If you read my 2 posts above, my thinking was precisely that if the proposals I summarized are implemented, then we may get around having to change the semantics, by implementing appropriate warnings and errors and introducing useful syntactic sugar, viz., for! i being sugar for global for outer i. All these can be done now (in 1.0.2) At least we might buy ourselves some time to get more data; the default can always be changed from local for to global for later if still deemed necessary, and in a semver compliant way.


#223

I have taught absolute beginners (other languages) and is indeed not really that difficult to introduce scope to them in a way they understand IF you can do it at the appropriate time (i.e. when introducing functions), and if it is a the version of scope as implemented in any other language:
You generally start with operators, variables and loops. Assigning something to a variable and then doing something with it in a loop is an extremely common pattern (I am convinced, even without checking, that the v1 change invalidated code in the majority of pre v1 tutorials). Having to explain that they have to use global to access the variable right in front of loop (for no apparent reason) is going to be difficult. The easy place to explain scope is functions: you have a separate piece of code that you are calling from another location, so there it is logical that the variables are contained to that part. You are also giving the values from your current location/scope needed by the function as parameters. One of the worst things of v1 scope then is that at this point you are going to have to explain that what they learned in the beginning (putting globals to get at the variable in front of the loop) does not apply if you put it in a function.

Some of the other suggestions here are even more complex and difficult to explain than v1. In my experience, if I start needing a lot of exceptions and lengthy explanations, I am usually on the wrong track (KISS principle). The original proposal by @jeff.bezanson is simple and consistent, and poses a very limited amount of incompatiblity.


Improving error messages for the scoping problem
#224

Adding my 2c to the teaching discussion, particularly fresh since I’m just wrapping up teaching an intro computing course for biologists in python.

My experience is that students, esp in an intro course, will take a lot on faith. All the syntax they’re learning is new, all the concepts are new. If you tell them that for is spelled global for, they’ll do it until you tell them to do otherwise. No one asked for example why it’s print(string) but string.split().

For the particularly curious students you can say, “great question - it’s not critical that you understand right now, but Google ‘scope’ if you’re interested in learning more, or stop by office hours.”

I don’t have a strong opinion on the underlying merits of the different ideas. I got bit by this early in the transition, but adapted quickly. Given how great the language design is generally, and how much time the core devs think and reason about the decisions0 (a fact which has become even more glaringly obvious in the last 6 weeks teaching python, which I used to think was fine), my inclination would be to trust the reasons for the design as implemented. More informative error messages are always great of course.


#225

Late to the party here. I just want to add some importance to the quote above. Packages are not the only things imported, datasets too. Big data analysts need to import data of 10G or even 100G into the memory; it will be a nightmare for them to keep restarting the program.


#227

This hasn’t changed and has always behaved like this; do syntax is just shorthand for passing an anonymous function to a higher order function. (Your code also doesn’t make much sense as far as I can tell—which m is supposed to persist outside?)


#228

Sorry I had to check what is OP saying! :confused: I will delete this message too to not spoil this topic…


#229

No problem! I wouldn’t worry about spoiling the thread or deleting anything, I think we’ve gotten the feedback we need at this point.


#230

Now that the debate seems to be cooled down, I would like to say that, when the devs feel that they have a clear roadmap and are confident with the eventual solution, I would love to read a blog post or some other kind of document explaining this point, and what we can expect in future versions of Julia, even before the changes are implemented.

Ideally, with hints for people who develop teaching materials, about how the issue might be approached to avoid misunderstandings.

I think that this will help to have users prepared for the changes, and will make the transition less messy.


#231

I’ve come up with a different solution that is non-breaking and solves the issue less drastically while retaining the benefits of the current design. I will write it up in a new thread and link to it from here.


#232

The proposal for my new solution is here:


#233

Coincidentally, I’ve suggested essentially the same idea here

@haberdashPI


#235

Let’s try to keep the conversations around this topic organized — the latest discussion is happening in the New scope solution thread. I’m going to close this topic in favor of that one.


#236