New scope solution

Jeff implemented this proposal.

1 Like

Making everything global outside of functions would be much simpler, but also much more breaking. The key feature of the proposal in this thread is that it does dataflow analysis to prove (with some caveats) that your code would get an undefined variable error. That allows, if we want, to leave all currently-working code alone and only automatically change variables to global if the code would have thrown an error anyway.

5 Likes

Makes sense. I do think this is a reasonable stop-gap for the 1.x series until a breaking change can be introduced in a major version. As @jlperla said, I appreciate the work you guys are doing amdist my kvetching. The way this rule works would be a very unusual as a ā€œdesired behavior,ā€ but, as a transitional step towards something everyone can live with, I guess it has its place.

1 Like

That is a very important point: If this is a stop-gap solution than people can certainly live with a compromise (I do!). But this is not the same to ask if the people want this behavior in general. Then I agree that the concept is somewhat convoluted and the 0.6 behavior was simpler (from my point of view). Having said that, I am absolutely pro the flow-based scope in top-level expression during 1.x.

1 Like

Feels a bit premature to be declaring that this is definitely happening. I just want to speak up for those of us that like the current behavior, and donā€™t want any change. And Iā€™m saying that as a non computer scientist AND someone that does a lot of teaching.

15 Likes

Thatā€™s why I didnā€™t do that.

I also prefer the current behavior, but I have to say that Iā€™m not very deeply invested in the behavior of the global scope, and I can understand why some teachers want to be able to teach the accumulator pattern in the REPL without doing a sidetrack on scope.

My real preference is that loops would inherit global scope at the top level according to the same rules by which they inherit outer scope in functions, but I have the impression that global scope is somehow so radically different from local scope in the implementation of Julia that this is challenging.

Granted, nobody has ever told me this. I just assume it to be true because I assume it is so obviously the correct behavior that it would be the default if it were not difficult to implement.

1 Like

The difference is that if implemented that way, setting variables in functions defined in the global scope would overwrite global variables with the same name ā€“ like they do for outer scopes when functions are defined elsewhere. I think most people see the global scope as different in this sense, no?

1 Like

I guess itā€™s a bit like having Python variables be nonlocal by default; if you also do that at top-level, theyā€™d be global by default. One important reason for making this default is that loops have their own scopes in Julia, unlike Python. So
Without this, youā€™d basically have to declare all variable uses inside loops as nonlocal for them to have effect outside. At least I think this is what the issue stems from ā€“ i.e., a semantic/behavioral issue, not an implementation issue.

2 Likes

Itā€™s probably my Python background coming through. I certainly donā€™t want functions overwriting globals without being very explicit in their intent, but top level loops? Seems natural to me (because Python, I guess). I also think the 1.0 behavior is perfectly acceptable, and making top level loops entirely in the global scopeā€¦ Ech. Itā€™s fine, too, even if I donā€™t prefer it.

But anyway, I see your point. Itā€™s likely that what feels natural to me is biased. Iā€™m used to being able to move code in and out of functions and have it work the same way in both places, but maybe this is illogical for others.

I think most people here agree that the behavior should be as similar as possible in these cases. (That was also my inroad here; studentsā€™ confusion about loops not working outside functions, where they were prototyping them.) I guess thatā€™s the impetus behind the discussion, really ā€“ how do we achieve that? Iā€™m also used to Python, and donā€™t instinctively see the need for separate scopes in loops ā€“ especially not treating them the same way as function scopes ā€“ but I guess thatā€™s part of the LISP legacy. (Though there were previously the soft scopes, of course.)

1 Like

I like the recently-implemented flow-based solution not just as a stopgap or compromise but in its own rightā€”I think the new behavior is intuitive and easy to explain, and will confuse or surprise fewer people than either the 0.6 or 1.0 semantics.

6 Likes

I think the ability of moving code from inside a function to the REPL is very useful. Absent a debugger, this
is the only way I know how to debug functions (apart from sprinkling with print statements).

So I would favor a scope rule which would be identical between functions and the REPL.

5 Likes

Itā€™s the same in most C-style languages. Loops normally open an new scope. Course, most C-style languages donā€™t have loops in the global scope at all. JavaScript does, but globals arenā€™t protected in JavaScript anyway. Itā€™s the wild west, baby.

1 Like

Sure. I just think in Julia, itā€™s part of the LISP legacy (as Julia is sort of a LISP) ā€“ like the let statement, for example, which is essentially syntactic sugar for function definition and application, explaining the scope behavior there :slight_smile: But, yeah, not uncommon to link blocks to scope.

2 Likes

In the flow-based version, what would be the canonical way of marking the a variable so that it behaves non-locally, to ensure its location doesnā€™t change unintentionally? One could use global, but that wouldnā€™t work if one wished to copy between local and global scopes. Something like x = x at the beginning of the loop would perhaps trigger the right flow analysis, but it would probably be optimized away. Perhaps the outer keyword could be used, analogously to the iteration variables?

1 Like

What if the GUI debugger were to arrive? Would that make you more accepting of the 1.0 behavior of loops?

yes

That makes me wonder to what extent this really is a debugger absence problem?

I suppose that for a large portion of users the only way to catch bugs in the absence of the GUI debugger is by copy-pasting function body and running it line-by-line in the REPL. But with 1.0 changes even that option is gone, so their workflow is paralyzed and hence they are so vocal about it.

Iā€™m speculating that if a GUI debugger arrives then it will take a lot of pressure from the 1.0 changes and it will seem like a minor issue at that point (which it really is) and with that, maybe we can even preserve the current behavior.

I see soft global scopes as a painkiller for a real sore ā€“ the absence of debugger. So if we treat the sore itself, then there is no need for painkillers anymore. :slight_smile:

TL;DR: Looks like GUI debugger + SoftGlobalScope.jl will keep everyone happy and would allow us to preserve the current 1.0 behavior of loops.

9 Likes

doesnā€™t help people whose primary platform for running Julia is Jupyter.

And this is also the first time Iā€™ve heard of someone pasting function bodies into the REPL for debugging. Not saying thereā€™s anything wrong with it, Iā€™m just not sure itā€™s a particularly common workflowā€¦ or is it?

It is for me. I will test lines in order to step through it in the REPL and then wrap them into a function. Itā€™s also the only way to diagnose misbehaving functions.

But I wouldnā€™t need to do that if I could step through a function with a debugger.

Put differently: For people accustomed to using a debugger, the most natural way to achieve something similar is to put lines into the REPL.

3 Likes