New scope solution

Internals & Design
205

Was there any case in which the 0.6 semantics were honestly a real obstacle for you? Or is it just philosophically distasteful?

Complex projects rarely do much in global scope, which is why I share @jlperla’s skepticism that soft global scope was ever an obstacle to “being a powerful, consistent, and well-designed programming language for complex projects.” The global scope semantics are mostly irrelevant to complex packages, and for interactive usage additional global declarations are inconvenient even if we don’t care about non-CS beginners.

(You’ll notice that complaints about this only really became widespread after 0.7 and 1.0 was released. In the 0.7-dev cycle, mostly it was used by package developers updating their code, rather than by interactive users or newbies, and package developers were barely affected.)

Anyone whose response to this issue is, “Well, I never write loops in global scope” (and there have been many such responses) is someone who isn’t affected by this one way or the other.

13 Likes
206

Keep in mind that one of the advantages of this proposal is that it’s compatible with existing 1.0 behavior. Anything that works in 1.0 wrt scope should also work with this proposal which means you (and I) can keep on pretending that loops always introduce a new, hard scope. I do find this complexity a bit distasteful from a language design standpoint, but I like that it allows people who like the simple, explicit behavior of 1.0 to keep writing code in that style; it just makes some cases that would have previously been an error behave a bit differently.

207

The fact that a let block in global scope now has identical scoping semantics to a function body is what has enabled the Rebugger. While none of the proposed schemes allow you to copy-paste function bodies into the REPL directly and have everything work, the current 1.0 behavior allows you to copy-paste function bodies into a let block. That is a very nice feature.

12 Likes
208

So this is the case where the 0.6 semantics of let blocks are a problem. But most of the discussion is about loops, so what is needed is a case when the 0.6 semantics of loops were an obstacle.
I think it makes perfect sense for let to be hard scope, like functions, when loops are soft scope (actually, I was surprised to learn just now that let was a soft scope in 0.6).

2 Likes
209

Agreed that the 0.6 semantics are imperfect. Does the proposed solution break this? It seems to make things even more consistent in the ability to move code around without swapping in global/local?

210

What about an escape hatch that simply restores the scoping required for Rebugger?

The point is that newbies and educators want top-level loops to “just work as naively expected” without annotations. Rebugger et al could just emit Meta.@__clean_scope_instead_of_dwim_scope let ... end instead of let ... end (macro name intentionally ugly, because it is only intended for code generation tools like Rebugger or Revise or IDEs).

A second point to consider is use in modules / packages, where top-level loops are used for setup of methods / globals. There, we absolutely don’t want accidental leakage of temporaries. My personal favorite would be to preserve the current scoping rules outside of Main (but I think a divergence between modules and REPL/script was deemed inacceptable?).

211

I hesitate to venture an opinion on this topic as this is beginning to take on the character of a argument about sports teams (it’s still much too polite to compared to a political argument). Unfortunately that generally means that people leave the discussion with, at best, an agreement to disagree.

I believe that everyone here can make a case for their preferred outcome and so, no matter what the final outcome is, roughly half the people are going to end up grumbling about it.

Personally a find the v1 scoping to be quite clear, easy to understand, sensible, well-considered and entirety frustrating.

The latter because it really poured some sand into the gears of my work flow. But work flows are not encoded in our DNA. They are habits we learned because we found them efficient within the environment we apply them. And so I can change my work flow to adapt where needed. Maybe I just add a global keyword inside loops, maybe I wrap more code in functions when working in the REPL. Whatever works. I’ll adapt and a week later I will have forgotten that I used to do things a little differently in the past.

At some point soon Julia will have a debugger and half of the reasons I work in the REPL at all will be gone, and I’ll adapt my work flow again.

Is scope an issue when teaching programming? Maybe. Maybe not. I guess it depends on the class. I also teach an in-house programming course with Julia. I jump straight into putting code into functions. No scope discussion until someone runs into a scope issue. Maybe that is an awful way to teach the subject, but so far no-one has died, so I guess it can work. When I was first taught to program 30 odd years ago, scope was just one more thing you had to know about right from the get-go, and as far as I can recall, no-one suffered a breakdown then either. There is simply more than one way to teach a topic and personally I don’t think any of them are fundamentally so much better than the others to justify that it should influence language design. Surely students who have the ability to get a degree can comprehend the basics of scope when they need to. It is by far not the most complex concept they will learn in a first programming course.

If the scoping rules are going to determine whether or not the end up using Julia or not, then I really don’t know how I would explain using packages to them. But I am not a professional teacher, so maybe I just don’t known what I’m talking about. There is near infinite opportunity for me to improve in that area.

If you are still reading at this point, you may be wondering whether there was a point to this post or not. Basically I would really like for the core devs to make a call on whatever criteria they believe are the most important and then that is that. Everyone else will get used to things and move on with their lives. Having an ongoing debate about something as fundamental as scope has much more potential to frighten off new users than any of the pros and cons of any of the options. No amount of debate and considering of opinions will make everyone happy. This is an excellent time for a well-considered executive decision.

15 Likes
212

Thanks for the input—and very well put.

That is how we’re going to do this. Just to be clear (in case it wasn’t already), the PR that Jeff has implemented is an experiment to see several things:

  1. How complex the implementation really is
  2. How breaking the design really is
  3. How it feels to actually use it

Jeff and I are still debating this and other designs on a regular basis and nothing is decided. This may not be what we do—there are some other approaches that have significant appeal and may be better, including something quite similar to what Jeff originally proposed in another thread on the subject.

7 Likes
213

I agree - I don’t like doing this either. Sometimes it’s necessary - personally, I’m planning to just teach students about scope earlier than I do when teaching python. It’s not so bad, and I actually anticipate this leading to less confusion overall. We’ll see - I’ll let you know after I run this syllabus for the first time :smiley:

I really wish people would stop making the blanket statement that educators want this. Some of them maybe. Not this one.

3 Likes
214

FWIW - I am a professional teacher, and I was nodding along the whole time reading your post.

2 Likes
215

I completely support this approach. Everything — variables, integers, whatever — has a lot of depth behind it that you don’t need to know at first. You can write 2 + (-3) without knowing what two’s complement is.

4 Likes
216

As I pointed out elsewhere I think you can recover function-level scoping in any scoping rule by using “immediately evaluate function expression” (IEFE):

(function()
    x = 0
    for i in 1:10
        x += i
    end
    x
end)()
2 Likes
217

Isn’t that essentially what a let statement is?

218

In Julia, a let acts like “non-repeated” for instead of an IEFE. See:

So, the rule for for applies to let as well.

219

Quick question about the new PR: how does it affect compile times of code inside regular functions? If the new static analysis does not get triggered at all on these, and only global scripts get slower compilation, I would be much less worried about this new design.

2 Likes
220

Correct, the analysis doesn’t run at all inside functions. So I believe the cost of it is very low.

2 Likes
221

Both let and for currently introduce a scope, so I don’t see what the relevant difference from IEFE is. Unlike for, let has the advantage that it evaluates to its last statement, so it may be used to get function-level scoping, e.g., this works:

let
    x = 0
    for i in 1:10
        x += i
    end
    x
end

No need to define a function.

4 Likes
222

We are discussing a new scooping rule of a for block. This automatically applies to let block unless @jeff.bezanson changes mind (unlikely, I think; see his explanation on why it’s preferable: Another possible solution to the global scope debacle - #194 by jeff.bezanson). For non-1.0 scoping rule for for and let, IEFE and let may be different.

1 Like
223

I might, in fact, change my mind… The problem is that

  1. @tim.holy points out that it is useful to have a top-level construct where code inside it works just like a function body.
  2. Nobody really has a problem with how let works currently; loops are the only issue.

So we might need to be open to handling loops differently.

9 Likes
224

I appreciate that you are flexible on this.

Indeed, it seems a one of the central points in the issue has been the friendliness to programming beginners. I think they learn let very late compared to for in their experience in programming.

1 Like