New scope solution

I’d be happy to just have it in scripts, really :slight_smile: But I guess it’s not really a feasible solution.

Just a question, not a proposal, request or suggestion.

Many other languages don’t have this issue because they require you to write var, let, type or def in front of every variable. Why was the design decision made in Julia to not use this approach?

1 Like

I also have the same question … i.e., if declaration and assignment can be distinguished syntactically, things seem much simpler
(like a := 1.0 vs a = 1.0 in Go), while not tedious to write. It can also help
avoid possible errors due to typo of variable names (e.g. unintentional declaration
of new variables).

2 Likes

I don’t know the history, but I think this was a choice between making the syntax more verbose or dealing with the consequences of occasional ambiguity. Since many languages, especially interactive ones, no longer require explicit declarations of new variables, users are now expecting this.

8 Likes

Context: I am a professional programmer now, but I started as a hobbyist and have no formal CS background, and I’m just getting started with Julia.

A few thoughts:

  • I thought the scoping rules in the loops in the REPL in 1.0 were weird. I read the docs. Now, I think they are fine, if a little inconvenient for interactive use. The thing I thought was weird is that using an accumulator pattern works inside of a function, but doesn’t work the same way in the global scope. However, it’s fine, once you understand that, in terms of scope, a loop block is the same as a function block.
  • I had to read over the current proposal’s rules (the rules I quoted) several times before I could begin to visualize how they would work. That’s probably not great. My feeling is that a rule like this would add extra cognitive overhead when reading code–and that’s definitely bad–even if it’s not so difficult for the compiler.
  • From the discussion, I’m not sure if this stuff only applies to loops outside of functions or what. The worst possible case is if global state could be implicitly modified from inside functions as a consequence of the changes being proposed. The alternative is also weird: different behavior inside and outside of functions.

If we can’t keep the current behavior, which is my vote, why not just have loops inherit values from the outer scope (which would be the global scope, in this case), similar to the way they inherit the outer scope in a function or other scoped block? I guess that means scope would mean one thing in function blocks and another thing in loop blocks, but that seems to be how most languages handle it.

I have one request, whatever happens:

Please do not change the way scoping works inside of functions. It’s already perfect. I don’t really care what you do with global scope, since I don’t put mutable things in it (except in the REPL, but I don’t mind having to write global a few times if I need to).

8 Likes

Happy Friday everybody. Here’s an implementation of this to try out: WIP,RFC: flow-based scope in top-level expressions by JeffBezanson · Pull Request #30843 · JuliaLang/julia · GitHub

18 Likes

Feels nice:

julia> t = 0
0

julia> for i = 1:100
           t += i
       end

julia> t
5050

julia> for i = 1:100
           t = 100
       end

julia> t
5050

julia> for i = 1:100
           if i <= 50
               t += i
           else
               t = 0
           end
       end
ERROR: syntax: Scope of variable "t" is ambiguous. Please explicitly declare it global or local.

Very polite error message too :yum:

10 Likes

I wonder if the error message should say where the declaration should go and how it should look, eg. in front of either use of the variable, and written as local t or global t.

“flow-based scope in top-level expressions”
do I understand right, this is only on the top level, e.g. in the REPL? Where else is top level?

And what exactly is the problem to which this new behavior wants to be solution?

I am afraid that there still remains problem with people which will intuitively expect same (and sane?) behavior as in local scope:

julia> function tst()
          t = 0
          for i = 1:100
             t += i
          end
          println(t)
          for i = 1:100
             t = 100
          end
          println(t)
          for i = 1:100
             if i <= 50
                t += i
             else
                t = 0
             end
           end
           println(t)
       end;

julia> tst()
5050
100
0
7 Likes
# if you past Stefan's code into file.jl you will get same output
julia file.jl

Top level is also in the global scope of a module (and by extension, any script).

Regarding this solution in general, having tested it, I still think it doesn’t feel right that the scope of a variable in a loop relies on static analysis of branches, but I guess it will make the common case easier for learning in the REPL, and people who know what they are doing shouldn’t be writing many loops at the top level in source files anyway (except for code generation, I guess).

So, while I think the concept is a little weird, it doesn’t affect function scopes, so it doesn’t really bother me.

3 Likes

Hello Everyone,
I’ve finally begun migrating from Python to Julia and really appreciate that the community is actively continuing to think about and passionately discuss these types of issues at this still early stage.

A large proportion of my workflow is interactive so I’m familiar with the arguments from primarily-interactive users, instructors, etc. However, after moving past some initial skepticism (stemming from prior experience with Python’s, R’s, Mathematica’s, etc behavior), I have now come to strongly believe that the current scope implementation in Julia 1.0+ is great “as is”.

That said, here are some suggestions for the community (apologies if any have been raised before) to facilitate better communication about this new feature:

  • Let's improve marketing: "In order to maximize code performance we've made some improvements to the scoping rules for the global environment in Julia 1.0". >> That will immediately get everyone on board, as it's hard to argue against better performance.
  • Improve the error message to "ERROR: UndefVarError: t not defined in current scope. Please refer to documentation about global and local keywords for the solution." >> That will provide a quick hint to those familiar with coding and enable online searches to quickly surface one's desired behavior for everyone else.
  • Improve documentation:
    • Explain scope rules for both the "global environment" and "function environment" up-front with simple "before" and "after (Julia >1.0)" behavior examples.
    • Show how wrapping for, while, etc blocks in let...end produces the "expected" behavior of legacy languages and is consistent with how functions also behave.
    • Explain merits of having a clean global environment. >> Especially useful because we can't re-define functions in Julia over an existing variable (while this is possible in other legacy languages, e.g. Python).
    • Explain merits of using functions as soon as possible.

I fully concur with @pablosanjose and @Liso’s points about simplifying scope rules.

  • Code block behaviour should work similarly in the global and function environments to eliminate complexity, ambiguity, and cognitive overload.
  • Since Julia will be used for the next 50+ years, let it define modern best-practice. We've already made the decision to migrate from other legacy languages so why should we continue using their old-fashioned behavioral paradigms?
  • Finally (though deeply respecting @StefanKarpinski and @jeff.bezanson's efforts to ameliorate the current drama with bold new solutions), I believe now more than ever that the old adage stands: "Just because you can, doesn't mean you should." ;-)
5 Likes

Didn’t you say you were from a Python background? :rofl:

2 Likes

Except that this is false. The change did not improve performance.

And you only write loops in global scope for things like interactive code that are not performance-critical anyway. (And that’s okay! Performance is not always a priority. The attraction of Julia is that it allows you to code for performance, not that it forces you to do so.)

That’s not the case with the 1.0 scope rules. They prevent you from taking the body of a (non-recursive) function and pasting it into global scope unaltered, whereas this used to work much more often.

1.0’s stability guarantees limit the opportunities to make a different scoping choice, and Jeff’s PR is a reasonable compromise. But let’s not make up fables to retroactively justify the 1.0 scoping change.

12 Likes

Thanks for the clarification re performance, etc.

I was just concerned about @Liso’s example showing that with the new PR, copy / pasting in the reverse direction from global scope to function now produces disimilar results.

In both 1.0 and the new proposed PR, there are cases where code would give an error in global scope, but work in a function. With the new PR there are fewer such cases, but not as few as there were in Julia 0.6.

3 Likes

Honestly, the more I look at this code sample, the more unsettling it becomes. Perl tried to exploit the notion of “context”, and I feel like that’s widely considered to be one of its biggest weaknesses, making code much less obvious.

The suggestion that the difference between t = 100 and t += i changing the meaning of t “feels nice” is…

Well, it’s certainly not descriptive of my feeling. Again, I don’t care if it’s just confined to this one place, but please don’t become comfortable with this kind of context-sensitivity. This should not be a trend.

17 Likes

This all looks spectacular, and I think a lot of people appreciate the hard work you guys have done on solving a tricky language design problem. Kvetching and gratitude are not mutually exclusive.

It seems like the intended behavior is: that the mental model people can apply to scoping of code within functions and in interactive environments is kept as consistent as possible, and it errors when ambiguous. I think this is all we could ever ask for.

How about encouraging people to put code in a function in the error? Like

ERROR: syntax: Scope of variable "t" is ambiguous. Please put the code block in a function, or explicitly declare it global or local

The reason I say this, is I am convinced that “really” using a global is almost never what an introductory user wants. They want to pretend they are coding inside of a big function - - and frequently copy the code into a function when they are done. If a beginner has to annotate with a global to avoid ambiguity, chances are that they should reorganize their code.

7 Likes

Out of curiosity: what were the reasons to go forward with this proposal instead of, for example, Jeffs?