New scope solution

This has been mentioned already I think: somehow it seems like there are two valid notions of let. There is a local one that is used only to get an output (which is I think how let is originally intended to be used in functional programming languages):

let a = 1
    x = a + 1
    x - 3
end

and then there could be a global one that is used to define global variables that depend on local ones:

let a = 1
    global g(x) = x + a
    global y = a + 12
end

I imagine this could be changed so that the former would be written as:

local let a = 1
    x = a + 1
    x - 3
end

and the latter as:

let a = 1
    g(x) = x + a
    y = a + 12
end

Similarly, the default for loop would become ā€œglobalā€ and for the local version one would write local for (this is pedagogically less of a concern as both local let and local for would not be used by beginners).
Then the rule would be: variables are global unless inside a function or in a block labelled with local. A (local) for loop is like a repeated (local) let.

I donā€™t necessarily think this is the best way forward (after an initial strong aversive reaction, Iā€™m starting to see the merit of Julia 1.0 scoping rules), but I wanted to point out this possibility.

2 Likes

I agree with your desire, but I think thereā€™s a bit more to it on the community side than just ā€œaccept and move onā€. As the discussions around this issue are old and beginning to repeat themselves with increasing frequency, I would expect the way forward from here to look something like:

  1. core team evaluates, internally, ALL criteria w.r.t. ease of use, community adoption, performance, simplicity, etc., etc. and decide on a solution
  2. the solution appears as a PR and set of criteria (indeed, this has already happened)
  3. if the core team is not happy with the implementation GOTO 1, else continue
  4. completed solution is scheduled for inclusion in a future Julia release
  5. solution appears in the wild and we all start this debate anew (or move on to the next one)

I agree with @braamvandyk that, at this point, itā€™s probably best for #1 to be left to the core team (and I, for my part, will refrain from any further ā€œarmchair PL designā€ going forward). I do look forward to helping out with #2, though, but I do think there is a fine line between dispassionately helping vet and validate the implementation of a solution and pushing the solution in a certain preferred direction. As for #3, I feel again this is the core teamā€™s prerogative, and feedback is probably best left to requests for such from them.

Step #4 has me most concerned. In a perfect world, the core team designs the perfect solution and designates it for release in accordance with SemVer principles. In the real world, thereā€™s obviously going to be a balance between how much we are willing to bend SemVer in the service of potentially having a better solution and not having to wait for (or prematurely declare) Julia v2.0. I expect the community will have a role to play in advocating for and assisting with any small SemVer violations that might be necessary. (e.g.: SemVer violations bite most when they cause packages to break unexpectedly. If, say, a solution were scheduled for release in v1.3 and the community worked overtime to discover breakages in public packages ahead of time, I imagine things will go more smoothly.)

As for Step #5, I wonder: has anyone given thought to a Julia community survey? It seems this is the season for language/developer community surveys (Iā€™ve completed 5 just this week), and Julia is somewhat of a conspicuous absence. Much of the hemming and hawing in this and other threads have been around what ā€œteachersā€ or ā€œstudentsā€ preferā€¦but without data, itā€™s all speculation and hearsay.

2 Likes

I ran into the soft/hard distinctions quite a bit. Not all the time, in fact rarely enough that I usually would not suspect them as the source of the problem so it would take some time to track down the resulting bugs. Which is why I was very relieved with the new scoping rules, I find them elegant and I also like the fact that I get errors instead of the language trying to figure out what I meant.

I would argue with this, eg

Changing scoping can have a lot of unforeseen consequences (especially in corner cases) in a complex language. So in a sense, my objection to introducing ad hoc rules is indeed philosophical: I am concerned that issues will surface later on, and for me the benefits just do not justify the risk of this (but of course, I am recognizing that other people have different preferences; I am just stating my own).

6 Likes

Thanks so much for taking the time to provide this great explanation.

After considering what you said, I still prefer the 1.0 solution, but I think you make a convincing case that the patch behavior is better than the 0.6 behavior. Itā€™s a little too ā€œsubtleā€ for my taste, in terms of readability, but I definitely get what youā€™re saying about the benefit of being statically resolvable.

I havenā€™t been able to read most of this, though Iā€™ve been through the more recent bits-- is there an opportunity to, at the REPL, ā€œmake it workā€ but just emit a warning when it does? Perhaps based on a flag that a REPL could use but which would be off for normal execution? (Thinking out loud naively here.)

In most of these cases being talked about, the warning would disappear when the user moved the code into a function. And the warning would make sure the user knew that if they wanted to really work with a global variable in normally executing code theyā€™d have to specify the scope, where the warning would become an error.

TL;DR: Does this PR affect to all constructs that introduce a local scope, including functions?

Sorry to resurrect the thread; it is not my intention to raise a new loop of repeated arguments, but to solve the specific question written above.

Motiviation of the quesion: most of the time the discussion has gone around the behavior of for loops (and comprehensions) at top level, because that is the situation where the current behavior is problematic in some circumstances (e.g. some teaching practices). The tests included in the PR (WIP,RFC: flow-based scope in top-level expressions by JeffBezanson Ā· Pull Request #30843 Ā· JuliaLang/julia Ā· GitHub) are also limited to for loops. But is this also affecting the scoping rules of other other local constructs?

My question is specially about functions, like this:

grow(x) = (a+=1)*x

In Julia 1.0, 1.1 this works only if grow is in a local scope that is also the scope of a, e.g. if both a and grow are defined inside another function. (For instance, grow might be an inner function used to calculate a geometric progression with a variable rate a.)

On the other hand, if grow is defined at top level, currently it fails. This looks like a sane behavior, since at top level you donā€™t have the control on the variable value of a that you would have in a function.

So, my question is: does the PR also change this behavior, such that grow would ā€œworkā€ at top level too? I have tried to look at the code of the PR, but it is beyond my understanding, sorry.

This still works the same.

Sorry, I donā€™t fully understand what this means. Does it mean that the PR does modify the rules of all local scopes (for, while loops, let blocks, functions, etc. - see the list in Scope of Variables Ā· The Julia Language)?

The PR does not modify anything about the behavior in a local scope. Itā€™s just special sauce for the global scope.

3 Likes

Thanks for the reply and your patience. But I have the feeling that I was not clear in the way I presented the question. Ok: nothing related to local variable changes, but variables from the parent global scope would be inherited for writing into local scope constructs, in some circumstances. This I already understood.

Iā€™ll put my question again more briefly. Is this a special rule in for loops as would look from the examples that have been discussed, or does it apply to any construct that introduces a local scope, including functions? Would this function work if the function is defined in top-level (and a is defined in the parent global scope)?

grow(x) = (a+=1)*x

Thanks again, and please bear with my dumbness if this was already clear from the previous discussion.

the two global scope threads are the longest on discourse :slight_smile:

2 Likes

Dear All,

I am working in a company where we use matlab as a tool for rather simple tasks than complicated programs. I am really struggling to replace Matlab with Julia, I am on a good way, though. But this scoping thing (and the time to first plot issue) are the main obstacles which prevents me to reach my goal. I mean from my colleagues perspective there is no serious reason to switch. However, they are willing to. But the two points as adressed above make them feel uncomfortable. I really really would like to see that Stefanā€™s proposal (first post) would be implemented in the near future. However, I have the impression that the Julia community has stopped thinking about that. Am I right by saying that?

Best regards
CS

1 Like

No. See recent activity:

3 Likes

They can just use Jupyter for interactive work, since IJulia uses soft scoping.

6 Likes

Thank you for the information. Have read now a lot in that huge thread. However, unfortunately it seems that the community is far from being in agreement in this regard. I am not able to follow and understand everything. In my (simple) world the scope issue shouldnā€™t be an issue at allā€¦

1 Like

It is very difficult for people coming from Matlab, since when you write a script in matlab things are only global if you mark them as such.

I think that the advice to use Jupyter notebooks is the best one, as people will find the scoping rules there completely intuitive. By the time they are ready to move to Atom/etc. they will have trained their brains enough to deal with the issue.

1 Like