The scoping rules feel unintuitive, and I feel more confused after reading numerous Discourse and StackExchange responses, such as Stefan’s explanation.
I come from MATLAB which has really straightforward and intuitive scoping rules, can someone explain why I should get different results between the two following scenarios? (I include them in a file at the REPL.)
bFound = false
for n = 1:10
if n > 5
bFound = true
println("bFound = ", bFound)
break
end
end
println("bFound = ", bFound)
which yields
bFound = true
bFound = false
as opposed to
function Test()
bFound = false
for n = 1:10
if n > 5
bFound = true
println("bFound = ", bFound)
break
end
end
println("bFound = ", bFound)
end
Test()
which yields
bFound = true
bFound = true
I just don’t understand the inner workings of programming languages enough to know why the best choice in the big picture of things results in obtaining different results here.
My suggestion: (1) stick with Jupyter for general scripting (which doesn’t have this issue); (2) put code in functions when you move to .jl files - which has other advantages. If you do that, you will never see this issue again.
As I understand it, at the first code, julia creates a local variable within the for-loop which is not related to the bFound outside of the loop.
they may have the same name but they are different variables.
restricting the scope of a variable to local scope can be done by defining them
within a function or a control construct. This way, we can use the same variable name more than once without name conflicts.
functions support local variables, but for, while try, let, and type blocks can all support a local scope. Any variable defined in a for, while, try, or let block will be local unless it is used by an enclosing scope before the block.
in order to tell julia not to create local variable with in the for block you call do one of the following:
define bFound whith in the for block as global (this is highly unrecommended dou to performance isuase):
bFound = false
for n = 1:10
if n > 5
global bFound = true
println("bFound = ", bFound)
break
end
end
println("bFound = ", bFound)
wrap the for block with let :
let bFound = false
for n = 1:10
if n > 5
bFound = true
println("bFound = ", bFound)
break
end
end
println("bFound = ", bFound)
end
wrap the for block with a function as you did in the second code, the function creates an enclosing scope, much like the let did.
julia> bFound = false
false
julia> for n = 1:10
if n > 5
bFound = true
println("bFound = ", bFound)
break
end
end
bFound = true
julia> println("bFound = ", bFound)
bFound = true
Same code in a file:
┌ Warning: Assignment to `bFound` in soft scope is ambiguous
| because a global variable by the same name exists: `bFound`
| will be treated as a new local. Disambiguate by using
| `local bFound` to suppress this warning or `global bFound`
| to assign to the existing global variable.
└ @ string:4
bFound = true
bFound = false
Thanks everyone for your responses, they’ve definitely been helpful!
My original (convoluted) question remains somewhat unanswered, but I’ll take that as a hint that while there are reasons for having the strange behaviour of scoping, they’re mentally unattainable for a beginner.
That said, I’ll mark Stefan’s 1.5 prospective Julia behaviour as a solution anyway.
functions, loops, try blocks, structs, and comprehensions introduce local scope blocks
when you see x = ... in a local scope:
if x is already a local, it assigns it
otherwise it creates a new local x
Is some part of this rule unclear? Or do you just find the result to be not what you’re used to? Intuitiveness and simplicity are often at odds with one another. Or put another way, mechanisms that humans find intuitive are often surprisingly complex and subtle.
I was typing out an example because I thought I knew where the confusion was coming from, but it turns out I don’t understand scoping either. Julia 1.4.2:
julia> module Foo
x = 1
println("From module Foo ", x)
function f()
println("From Foo.f ", x)
println("From Foo.f after assignment ", x) # I know I haven't done any assignment here, see the next example for context
end
end
From module Foo 1
Main.Foo
julia> import .Foo
julia> Foo.f()
From Foo.f 1
From Foo.f after assignment 1
This works, but just add one more line and it doesn’t.
julia> module Foo
x = 1
println("From module Foo ", x)
function f()
println("From Foo.f ", x)
x = 2
println("From Foo.f after assignment ", x)
end
end
From module Foo 1
Main.Foo
julia> import .Foo
julia> Foo.f()
ERROR: UndefVarError: x not defined
Stacktrace:
[1] f() at ./REPL[1]:5
[2] top-level scope at REPL[3]:1
There is no ambiguity. In your case, x is a local variable and you are trying to read it before it is defined. If you wanted to use the global x you write global x.
One particular fact that might not match your intuition is that a given identifier (ie variable name) can only have one meaning in a given scope block: if it’s local anywhere in a block, then it’s local everywhere in that block. You might be expecting x to be global the first time you access it and then become local after it is assigned but that cannot happen. Since x = occurs in the function body, x is local to the function both before and after that assignment.
By now I understand the scoping rules I believe, and @scimas’ code makes sense to me, about Julia looking at the entire function first, instead of say Matlab which does some things statically and other things dynamically.
Speaking of Matlab, my confusion is just as to why the scoping rules for Julia are the way they are. I come from Matlab where the scoping rules make so much intuitive sense to me, and I’ve never had a problem with it. But knowing the ideologies of Julia creators, I have faith that there’s a bigger picture as the foundation for why the Julia scoping rules are the way they are.
My question was two-fold, i.e. 1. How does scoping work and 2. What is the motivation behind such rules. #1 is answered, #2 is still in the air for me, with some unclear hints to the answer.
I apologize for not being clear in my long-winded post haha!
I am not sure that is the right model to have. I think the right way to think about it is that Matlab has scripts where julia doesn’t (except, Jupyter notebooks, which can serve a similar purpose). This isn’t exactly true, but it is a close to truth to form a mental model. In julia, what look like scripts (e.g. the .jl files) aren’t. They are simply text which can be included like anything else into a session in whatever order you wish (and doing whatever you want in the middle, e.g. using the repl).
That is why in matlab in your .m files you have to manually say global on a variable, and everything else is local to the script. In julia, anything at the top level is a global.
In fact, the global keywords in the language are completely different. In matlab you declare a variable as global in your script because otherwise they are local to it, whereas in julia you provide the global in a function/loop/scope to tell the compiler you want to refer to a top-level variable rather than create a new local one.
The main place that the “everything top level is global” gets you is in in the loops. This doesn’t happen in matlab because loops don’t introduce a new scope. This is actually a major benefit of Julia for bugs, reasoning about code, etc. but has this confusing scoping downside.
In Julia <= v0.6, in Jupyter notebooks the whole time, and in the REPL julia >= 1.5, there are special scoping behavior to make this downside basically non-existent. You will find working in Jupyter entirely intuitive, and soon the same for the REPL. For jl files, you will still want to wrap things in functions… but there are plenty of other reasons to do that.
I don’t think there is any ideology here, just tradeoffs. The only ideological decision distinct from matlab is that loops/comprehensions/etc. introduce a scope - which I think is a good thing.
There are downsides in the scoping approach taken in the v0.6/jupyter/v1.5 REPL approach as well - though these typically confuse more advanced users rather than beginners. Rehashing those downsides wouldn’t be helpful, but to suffice it to say the chance in v0.7 in Julia was legitimate reasons.
It is all very confusing, especially where people come from matlab where the global keyword means something completely different. The issue is was enough that the language designers brought things back to have consistent scoping in the REPL to match jupyter.
My suggestion is simple:
Use Jupyter for “scripts” where you are doing exploratory top-level code. And intend to copy/paste that code into functions eventually. In 1.5 you can do the same in the REPL as well.
Put all loops/etc. inside of functions in the .jl files
If you follow (1) and (2) you will never see this issue again. The scoping will be completely intuitive for normal stuff.
Forget everything just discussed in this thread. There are much more important things to learn and in in practice you won’t need to think about this unless you do very advanced programming in julia.
If you are interested really interested in the history, here are some links:
Way back in 2012, when the language was in its infancy, there was a discussion about scoping that eventually motivated “soft” and “hard” scopes, which Julia had until 0.6:
This was by and large intuitive, but had some corner cases. Scope simplified conceptually for 1.0:
but some people still found it unintuitive:
so
was introduced to make a particular use case easier.
You will notice that a lot of thought went into scoping rules. If you want a deeper understanding, I recommend working through the code examples people posted in various discussions above (and some others you will find from there); I found it really instructive.
Generally it is not easy to define scoping rules that are intuitive (“do what I mean”), yet easy to reason about (including corner cases), especially in languages that don’t have different syntax for assignment and introducing new variables.
IMO the important thing is not whether scope behaves in a way that users from some other language will find intuitive, since Julia users come from variety of languages with different solutions to scope; but whether scoping is easy to understand and apply in practice after reading the relevant chapter in the manual. Personally, I think Julia’s current approach is a rather nice practical solution.
Ah that makes a lot of sense. I’m further coming to process that things that may look the same in different languages can be quite different.
That makes sense. When I mentioned ideologies, I was referring to the more general purpose of Julia, to place the greater powers of programming into the hands of those who want easier ways to implement things while getting high performance out of it, particularly for large scale projects. And that filters down to the motivation for defining scoping rules regarding loops, which I have been asking about and can accept.
That’s an interesting observation, I was unaware of that, thanks for sharing.
Haha, yeah well I have quite a few questions in my experience with Julia that I haven’t chosen to ask about. I’ve just chosen this scoping one because it seems like other people have asked about it but the answers seemed pretty involved so I decided to ask for an answer for a beginner programmer, which is what I am.
Sorry if I was unclear, what I meant was that the downsides that @Tamas_Papp described in his response above - which triggered the change from Julia v0.6 change - are typically issues for more advanced programmers whereas the older behavior didn’t seem to confuse new users. “A priori”
I don’t think this was obvious and it only became clear after the Julia 1.0 was released (since beginners understandably didn’t engage in the long beta/release candidate stage prior to the release).
Anyways, glad that things are starting to be more clear. What I can tell you is that other than the issue you have stumbled on, scoping in Julia is far superior to Matlab/Python in nearly every other respect - especially when it comes to the possibility of writing efficient code with fewer silent bugs.
Ah, thanks for clarifying that as well. I was wondering what was going on behind the scenes that you were referring to.
And yeah, I’ve started reading the discussions in the links @Tamas_Papp shared, and it’s expanding my understanding. The things available upon a Google search lacked context for me I suppose.