New scope solution

[StefanKarpinski]

Hopefully in 1.1 which should be out in mid December.

[Ronis_BR]

Guys, I cannot contribute in terms of the design due to lack of knowledge. I just want something simple to use and debug

However, I am seeing that these changes can lead to future problems in terms of design, optimization, etc. (right?) IMHO, even though the v1.0 was promised to be stable and things will not break, please, always consider the downside of it.

For example, if you add a modification which is not good to keep backward compatibility, think about the pros and cons. Julia is a very new language, and I think it will be ok to introduce some deprecations here and there for the greater good (of course I am assuming that there is not an official contract or something legal…).

Anyway, I am thinking that the solutions to add something like a #pragma to change the behavior in REPL resembles too much to the IMPLICIT in FORTRAN, which is not good in my point of view. If I had to vote, I would always opt for the scenario in which there are only one behavior in the core of the language, even though it is not the scenario I prefer.

@StefanKarpinski

Just to see if I understood, the x in the following codes is local right:

x = 0
for i = 1:5 
    if rand() < 0.5
        x += 1
    else
        x = 1
    end
end

for i = 1:5 
    if rand() < 0.5
        x = 1
    else
        x += 1
    end
end

[StefanKarpinski]

@Ronis_BR

Anyway, I am thinking that the solutions to add something like a #pragma to change the behavior in REPL resembles too much to the IMPLICIT in FORTRAN, which is not good in my point of view.

We’re not doing the pragma or flag thing. You shouldn’t have to know what settings the language is using to understand what code means.

Just to see if I understood, the x in the following codes is local right:

Both of those examples would be errors and require you to explicitly say local x or global x because there’s on branch that has a read first and one branch that has a write first.

[stevengj]

@mohamed82008

May I ask what is the rough timeline for this change? I am asking because I want to give a Julia tutorial in my school and if this change won’t take too long, I will wait.

Note that IJulia already enables the SoftGlobalScope transformation (i.e. Julia 0.6-like behavior for global scope) by default.

[Liso]

@StefanKarpinski

You shouldn’t have to know what settings the language is using to understand what code means.

@stevengj

Note that IJulia already enables the SoftGlobalScope transformation (i.e. Julia 0.6-like behavior for global scope) by default.

[StefanKarpinski]

Yes, and that’s not an ideal situation.

[Vic]

I’m one of the users who expressed his confusions about loop scope not long ago (and got helped to understand it, many thanks).
I could give feedback for this and the other solution, from the perspective of a (not long ago) beginner to Julia.
I had trouble understanding this kind of loops:

a=1;
while true
   print(a)    # optional
   a=2 
   print(a)    # optional
   break
end

Presence or removal of one or both of the (innocent) print(a) had drastic effects on behavior of the loop, and understanding it was not at all a simple top-down following of written instructions.

The most recent solution presented this topic

@StefanKarpinski

• if the first use of x on every path is a read, then x is global
• if the first use of x on every path is a write and there’s a read somewhere, then x is local
• otherwise x must have an explicit local or global annotation

also make the behavior depend on presence of a read, and to understand that, the rules are not (much) simpler than currently in v1.0 , IMHO.
But I appreciate the efforts to try to come with simpler rules.

I also believe that even for a non-beginner (or not complete beginner), it’s useful
to be able to add/remove test printing/show statements without affecting the main results, and to have simpler rules, not requiring scanning of the branches inside the loops.

So I think the previous solution by @jeff.bezanson is better in these respects, even though it is more breaking.
My understanding is that, with that solution, the while above works in global scope: assigns to the global a regardless of any printing statements, and regardless of whether a=1 is present above the loop.

As an addition, I liked the suggestions there to allow (as optional) keywords local while (same for all other currently local constructs, except functions) that would make the assignments inside the loop by default local (but without depending on presence of reads).
Inside both while and local while, one could override the behaviour with, respectively, local a=2 and global a=2, which, for ease of reasoning, should only apply to code after these statements.

I’d also very much like if the rules for scope constructs written inside functions, or other local scopes, match the above rules – perhaps in future.
This way, we’d have a single, explicit and simple set of rules everywhere.

[RaulDurand]

Hmmm… Well, why loops need to create local bindings at the first place?
I only get the case for counters.
It is not clear to me why loops behave different than if clauses in terms of variable binding?

[pablosanjose]

I also want to cast my vote to @jeff.bezanson’s previous proposal over the one in this thread… if we indeed need to depart from the 1.0 rules (which I like best of all).

I truly appreciate the great effort that is being devoted to this problem, and I know that this decision is not easy. However I cannot understand why assumptions of people coming from different scripting languages should have so much weight, to the point of introducing such convoluted, subtle, contextual and long-range rules to my favourite language. I have the feeling this particular change could potentially complicate Julia’s future (compile-time) performance, and its overall simplicity and elegance. A single print can change the scope of a variable? And this just for the sake of teaching convenience and compliance to conventions? Call me inflexible, but that’s a no thanks from me. FWIW.

[jballanc]

I think I understand and can communicate now what it is about this proposed solution that bothers me so much. I’ve been trying to break down the stated “rules” and recombine them into the simplest possible statement, and what I’ve come up with is:

• Variable’s are resolved in local scope
• If a variable is not present in local scope, it is resolved in global scope
• …oh, yeah…and variables within certain special scopes are statically resolved, otherwise Julia names are dynamically bound

The first two rules seem extremely straight-forward until you realize that they only work if we are, sometimes and for certain purposes, resolving variables eagerly/statically. In other words, this solution creates a sort of Late static binding semantic (for scope, but not value) in Julia, which seems rather out of place with Julia’s otherwise late binding. (Not to mention that, if we are following branches and erroring when sibling branches yield ambiguous resolution semantics, we’re not even truly doing late static binding but something like “dynamically late static binding”.)

I still think it would be best to try and not be too clever. What would be wrong with a “variables resolve locally and fall back to global scope” type rule, keeping Julia’s late binding semantic? Yes, you could contrive of weird examples with this rule where different branches result in resolving a variable either globally or locally based on some input (or random), non-deterministically. But Julia already has a late binding semantic that eliminates the ability to determine correctness of operations. e.g.:

function foo()
    if rand(Bool)
       bar = "string"
    else
       bar = 0
    end
    for i = 1:10
        bar += i
    end
end

This function sometimes works and sometimes doesn’t because the type of bar in the for loop isn’t known until the for loop is executed. So, what I’m saying is, would it be so bad if whether bar was local or global also was not determined until the for loop was reached?

Yes, this would be a “gotcha” of a sort, but every language has “gotchas”. What really matters (in my mind) is how hard it is to explain the “gotchas”, and I feel like the “gotchas” the solution proposed by this thread would generate would be much harder to explain.

[StefanKarpinski]

@jballanc

So, what I’m saying is, would it be so bad if whether bar was local or global also was not determined until the for loop was reached?

Yes, yes it would. It would very bad and far worse than even the somehwhat statically unpredictable 0.6 behavior. What execution path is taken and therefore what value is assigned to bar is a question of what the code does, which is inherently unpredictable and dynamic in any Turing complete language. Whether bar is global or local is a question of what the code means which, as a very broad principle, in Julia is never dynamic or unpredictable—meaning of code is always statically decidable. You don’t even need a tables to know what is a type vs value or function vs macro. The top-level default scope behavior was the last bit of unpredictable meaning left in the language. And people complained about it over and over again. So we removed it. Of course you know how that went.

Your argument seems to me like it basically comes down to “the language allows runtime exceptions so we can make anything as unpredictable as we want.” Which seems very much like throwing the baby out with the bath water.

[Azamat]

Could someone, please, summarize or point to a summary of what was problematic about behaviour of loops in 0.6 that motivated switch to the current behavior in 1.0?

[StefanKarpinski]

What does the following code do when evaluated in global scope?

for i = 1:n
    t += 1
end

Does it:

a) increment a global variable called t
b) cause an undefined variable error
c) who knows, could be either one?
d) something else entirely.

[PetrKryslUCSD]

It reports that n is not defined

[StefanKarpinski]

Assume that n is defined and equal to 10.

[Liso]

Is there any language other than Julia where variable in global/outer scope is not incremented?

[jballanc]

@StefanKarpinski

Your argument seems to me like it basically comes down to “the language allows runtime exceptions so we can make anything as unpredictable as we want.” Which seems very much like throwing the baby out with the bath water.

I’d like to think my argument was more along the lines of “there’s the possibility of a variable escaping local scope into module scope…which ain’t really all that bad”. If we were polluting a true runtime-global name scope, I’d be more cautious about suggesting such a fix. (Again, taking the example of first, it is possible you might accidentally re-define first as a variable for code in a module, but you wouldn’t break the Julia runtime unless that module also imported Base.first, and you’d only be doing that if you had some intention of re-defining first with that module in the first place.)

Actually, originally I was going to suggest eager-name-resolution with dynamic-value-rebinding similar to what Clojure does, but I figured that would break far more code (though it would allow the simplification of scoping rules I was aiming for).

I suppose, in truth, what we’re talking about is not altogether different from Hindley-Milner style type resolution…except we’re resolving scope instead of type (and in a dynamically typed language… ).

---

Ok, on a more serious (hopefully productive) note: how are we defining “first use”? Is that semantically independent position in the parse tree? A contrived example:

x = 1
for i = 1:10
    atexit(()->(x = 10))
    x += 1
    println("x is $x")
end

Compare this to the equally contrived example:

foo(f) = f()
for i = 1:10
    foo(()->(x = 10))
    x += 1
    println("x is: $x")
end

Currently, on v1.0.1, the first throws an UndefVarError while the second prints x is: 11 10 times in a row. What’s the behavior under the new rules?

[tanhevg]

Not sure if this has been considered before, but here is an idea. Can we change the behaviour of REPL and scrips, such that all top level variables share a top level local scope, as opposed to global? Essentially, what I am proposing is equivalent to implicitly wrapping the entire script or REPL session in a huge let ... end block . Behaviour of modules does not change.

This would fix the issue with incrementing the outer variable from within the for loop, which started the original discussion. It is also simpler to reason about than all the solutions that involve analysing the read/write access to the variable across different code paths. Also, this is simple to manage across multiple versions. Like have a command line options for local and global top level scopes, and make global the default in 1.1 and local the default in 1.2

[mlhetland]

Explicitly wrapping in let is the solution I suggest to my students, anyway. Explaining that this happens implicitly would be quite straightforward, along the lines of having stuff wrapped in Main. (Though I’m sure there are lots of problematic cases – just can’t think of them at the moment.)

[mauro3]

This cannot work as eval only works at the global scope, and the REPL relies on eval.