Remove the soft scope altogether and make it global

Internals & Design
,
41

Hi Benny,

I think I should clarify a little. I am not against having the option to include files. To this day I hate it that there is no include in HTML or JavaScript. If you write multiple scripts that access the global scopes, you know that you should be careful to avoid variable clashes, and convert some of them into functions. There is nothing wrong in writing multiple scripts as long as you are careful about the variable usages. There is an inherent risk in it which those who write are aware of. But because of hard scoping, I am having to redeclare all the variables in every for loop.

My point is it is a bad trade-off.

42

Revise is a fundamental tool in Julia development. And with some practice of the most convenient Julia development workflows the problems you are having with scopes will be a non-issue.

I think Julia has some early barriers that cause initial bad impressions. But those are by far compensated by the qualities, which one grasps with a little more experience. From that point of view of early and not-performance-critical prototype or scripting it has no advantage over other interpreted languages.

2 Likes
43

I would suggest experimenting a bit more with the current available options before holding your breath on v2.0, which will probably not change the scoping rules.

You mentioned let in one of your posts, doesn’t it work well for your workflow? like wrapping your entire script in a let block allows you to not have to insert global declarations.
If you workflow involves include, an alternative is to use the REPL.softscope function:

import REPL
include(REPL.softscope, "script.jl")

At the command line, it could look like: julia -e 'import REPL; include(REPL.softscope, "script.jl")'

2 Likes
44

Wow! This might be the one I need. Thank you!
Btw, is there a way to enable soft-scoping by writing some statement at the beginning of a file?

let blocks delete all the variables after the end when running scripts. Makes it difficult to debug.

45

Containing code in functions is encouraged in Julia, too, so we seem to agree on the importance of having a local scope whose variables are local by default, isolated from the variables in the global scope. Then you are in effect proposing that for-loops shouldn’t introduce a new (local) scope, like begin and if blocks currently. That’s not an entirely bad approach, I know Python does it that way. However, that causes an infamously unintuitive issue with capturing variables, here’s a short Python example: [f() for f in [lambda:i for i in range(3)] ]; [0, 1, 2] is expected, but since i is not a new local variable per iteration, [2, 2, 2] is the result.

Besides, it means for-loops in global scopes are still susceptible to the locals-as-globals bug I described earlier. You’re right that being careful about variable clashes will still work, but it’s unreasonably hard to be so careful across a large project, especially with multiple people contributing (can’t read each other’s minds to avoid naming clashes).

When copying from a local scope, it really does just make more sense to paste it in a local scope (let block). For example, if you want to test out several code segments from different functions, you probably don’t want them to collide in the same global scope and have to restart the session. If you want to see what variables’ values are, you could do some println-debugging in your snippet, or you could put some @debug lines in your source code and “activate” their printouts (see Logging module in standard library), or you could use Debugger.jl for stepping and breakpoints.

1 Like
46

you can do something like:

# script.jl
import REPL

macro softscope(ex)
    esc(REPL.softscope(ex))
end

@softscope begin
    i = 1
    while i < 3
        println(i+=1)
    end
end

and in the REPL (no need to use two-arguments include anymore here) :

julia> include("script.jl")
2
3

julia> i
3
2 Likes
47

I think those are possible options, but they derail a bit from the fact that there is a much better workflow. Here I have a file test.jl, which contains a function, which has the so discussed loop:

Using Revise you can includet("./test.jl") (note the t) and track the changes to the file. Running the function after the change automatically tracks the code.

Additionally, you can select the code and copy/paste, or Control-Enter, into the REPL, and all that works because of the soft scope of the REPL.

All this may not be equal to what one is used in other languages, but it is convenient enough, maybe more in some aspects. With the additional bonus that the function in case can then be benchmarked with true production-time performance.

3 Likes
48

To make this even more confusing, as @pfitzseb said vscode also executes your .jl files with the softscope as you go through with the inline execution. Having code in .jl vs stored in a jupyter notebook is not sufficient to know what will happen in the scoping - especially during debugging/interactive use.

Essentially every usage (e.g. vscode inline, jupyter, repl, etc.) uses the softscope, which you find intuitive, EXCEPT including with include("myscript"). There were many legitimate reasons why the distinction was introduced but it has been slowly chipped away to make things more intuitive for most users. Nevertheless, at this point if you use loops in top level code you need to understand how the code is being executed to understand what the behavior would be. Copy/paste and file extension isn’t enough.

One way to deal with this is to primarily work in the vscode editor with <shift-enter> when testing and debugging, and then always make sure to wrap any loops in functions when you are done.

But the sooner you start wrapping any loops in functions the better, and for many reasons outside of this confusion. If you do, you will never hit this issue again. Regardless, if you ever put a global or an explicit softscope call in your code in the meantime, chances are you should rethink your workflow or code organization.

3 Likes
49

It’s not a question of performance but semantics. In version 1.0 of Julia, the soft scope for loops wasn’t applicable anywhere. Eventually, so many people whined about it that it was changed in the REPL for ease of use. However, Julia is committed to semantic versioning, which means that we don’t introduce new language changes that break old code if that code was using the language’s public, documented interfaces.

Switching julia to use a soft scope for loops could change the behaviour of old scripts written for version 1.0, so it was not allowed to make that change. It was decided that we could at least make the change in the REPL, but I think that was a bad compromise because it means that the language’s semantics depend on where you’re executing the code.

3 Likes