Feature request and Brainstorming , global let (issue #37187)

This is a feature request proposal Brain Storming

The proposal (initially)

  1. add construct global let to julia
global let 
    ...
end 
  1. Scripts (without) module declaration should assume being inside a global let
  2. add our and my as synonym to global and local

Inside a global let all variables and functions are global by default,
unless explicitly declared local

What does this fix and how does it help

  1. allow local variables in a file (file scope)
  2. make it easier to use the repl
  3. bridge or eliminate the gap between file scope and repl scope
  4. promote variable declaration before use (variable autovivification considered harmful)

I will explain below how this is this case

first some quirks

Currently this code below generate an error inside a file raise an error and a warning

s = 0

for i = 1:10
    t = s + i
    s = t
    println("$s")
end

println("$s")

┌ Warning: Assignment to `s` in soft scope is ...
ERROR: LoadError: UndefVarError: s not defined

this code still raise an error

local s = 0

for i = 1:10
    t = s + i
    s = t
    println("$s")
end

println("$s")

ERROR: LoadError: UndefVarError: s not defined

and this code still an error

global s = 0

for i = 1:10
    t = s + i
    s = t
    println("$s")
end

println("$s")

┌ Warning: Assignment to `s` in soft scope is ...
ERROR: LoadError: UndefVarError: s not defined

Now put the code inside a let, and the problem is solved

let 
    global s = 0

    for i = 1:10
        t = s + i
        s = t
        println("$s")
    end

    println("$s")
end 

And s is global and available for me outside the let all 3 variations will work, the difference is the visibility of s outside the let

I understand that the Julia way to fix this was to add global or local next to the first use of s inside the loop
and if I want s to be local that would not be so bad, but for a global s this is very counter intuitive

Mimicking file scope

Example

# file one.jl

let 
    global x =1 
    local y = 2 

    local function addx( a )
        println( x + a)        
    end
    global function addy( a )
        println(y + a)
    end

    addy(3)
    addx(3)
end 

#file two.jl
include("one.jl")

println(x)
addy(10)
#println(y) #raise error
#addx(10) #raise error

If we have global let file one.jl would look like

# file one.jl

global let 
   x =1 
   local y = 2 

   local function addx( a )
       println( x + a)        
   end
   
   function addy( a )
       println(y + a)
   end

   addy(3)
   addx(3)
end 

And would work just the same
Also if global let becomes implicit, file one.jl could look like this and still work just the same, allowing local variables that are file scoped

# file one.jl

x =1 
local y = 2 

local function addx( a )
   println( x + a)        
end
   
function addy( a )
   println(y + a)
end

addy(3)
addx(3)

The case for my and our
my and our are much shorter than global and local,will make the code look nicer and might motivate more developers to declare that variable scope before use

# file one.jl

x =1 
my y = 2 

my function addx( a )
   println( x + a)        
end
   
function addy( a )
   println(y + a)
end

addy(3)
addx(3)

or

# file one.jl

our let 
    x =1 
    my y = 2 

    my function addx( a )
        println( x + a)        
    end
    
    function addy( a )
        println(y + a)
    end

    addy(3)
    addx(3)
end 

Show stoppers
module must be declared at the top level and cannot be declared inside a let , I am not sure what the impact of allowing module declaration inside lets would be, and this is a big problem

But if making global lets implicit is impossible for some reason I could not preview, I still think that having the syntax variation of global let , my and our is still worth it

our let 
  my y = 1
  addy(a) 
      a + y 
  end
 ...
end 

vs 

let 
    y = 1
    global addy(a)
        a + y
    end 
    ...
end

Anyway, hope to get some feedback, do you think global lets is a good idea, does it have a chance?

I thought a bit more about this, and I think I found a fix for what I thought was the show stopper
modules simply should behave the same as global let , this will unify all scoping rules

  • inside modules
  • inside files, without a module declaration (again files will assume a global let)
  • inside the REPL

To summarise my request

  1. add global let where inside the let variables and functions are global by default unless explicitly declare local
  2. add our and my as synonym to global and local
  3. modules, files and the REPL will assume to wrap all its content inside a global let (this will allow to declare private/local module variables, file scoped variables and unify the scoping rules between all 3)

I’m a bit unclear about these motivations:

  1. allow local variables in a file (file scope)

Ok, but why? What’s the benefit of this? Presumably when you say “local variables in a file” you mean a variable that behaves as if you had wrapped the file in a let block and declared the variable in the let block. So it would be accessible from any functions within that let block but would not be accessible from outside the let block.

  1. make it easier to use the repl

How does this make it easier to use the REPL? What concrete problem in the REPL does this solve?

  1. bridge or eliminate the gap between file scope and repl scope

In what way? Again, what concrete problem does this address? What is the gap between file scope and REPL scope that this is eliminating?

  1. promote variable declaration before use (variable autovivification considered harmful)

Julia doesn’t do any autovivification. If you assign to a variable in global scope, that creates or updates a global. Unlike e.g. Perl (presumably where you’re coming from since you propose my and our as modifiers), accessing a global that doesn’t exist does not “autovivify” it—it’s an undefined variable error. If you want to assign to a global from a local scope, you need to declare it as global.

2 Likes

Before I explain myself, I want to make it clear, I think this change is mostly aesthetic and syntactic
it doesn’t really add any new features to Julia

It just makes using some features easier
You can today, wrap all the code inside your module inside a let have have local module variable
Also today you can wrap all the code inside you file inside a let and have local file variables that will not be accessible to other scripts that include your file

So this is mostly syntactic and convenience

Yes exactly, and I understand its not a big thing, just convenience, and check the below

# example 1
    global s = 0

    for i = 1:10
        t = s + i
        s = t
        println("$s")
    end

    println("$s")

# example 2
    s = 0

    for i = 1:10
        t = s + i
        global s = t
        println("$s")
    end

    println("$s")

To me, example 1, look more natural, yet today it will raise an error unless wrapped inside a let
if global let becomes the default behavior for file (and modules) example 1 will just work, and I think this is a good thing

If global let become the default behavior for files, modules and REPL, you wont need different scoping rules for the REPL and file, I think this is a good thing, less cognitive burden
I can still simulate this today, if I wrap all my code inside lets and carefully label all my functions and variables local or global, it is just a convenience I doubt it will break any code

I agree with you, I misused the word autovivification, my bad
And thanks for taking the time to check this :slight_smile: really appreciate it

2 Likes

This is in the eye of the beholder; also in 1.5 you don’t need the global at all, just use

s = 0
for i = 1:10
    t = s + i
    s = t
    println("$s")
end
1 Like

I am running 1.5.2 and I get an this error

PS C:\dev\lang\julia> julia --version
julia version 1.5.2
PS C:\dev\lang\julia> julia .\testscope.jl
┌ Warning: Assignment to `s` in soft scope is ambiguous because a global variable by the same name exists: `s` will be treated as a new local. Disambiguate by using `local s` to suppress this warning or `global s` to assign to the existing global variable.
└ @ C:\dev\lang\julia\testscope.jl:4
ERROR: LoadError: UndefVarError: s not defined
Stacktrace:
 [1] top-level scope at C:\dev\lang\julia\testscope.jl:3
 [2] include(::Function, ::Module, ::String) at .\Base.jl:380
 [3] include(::Module, ::String) at .\Base.jl:368
 [4] exec_options(::Base.JLOptions) at .\client.jl:296
 [5] _start() at .\client.jl:506
in expression starting at C:\dev\lang\julia\testscope.jl:2

And actually if my feature got accepted this code will run as if it was

let 
    global s = 0
    for i = 1:10
        t = s + i
        s = t
        println("$s")
    end
end 

but you will either write it

global let 
    s = 0 
    # s becomes global by default, and will be available out side the let 
    # to make it local you would need to explicitly 
    # specify this by using the local keyword, or hopefully the shorter my keyword
    for i = 1:10
        t = s + i
        s = t
        println("$s")
    end
end 

or just (if global let becomes default or implicit inside files and modules)

s = 0
for i = 1:10
    t = s + i
    s = t
    println("$s")
end
 

I am confused now, originally your proposed this feature to

but now you are testing it in non-interactive mode. Are you aware that the two have different scoping behavior? See

Yes, exactly.
A global let will unify the rules

If a file have a global let behavior by default, and the repl have a global let behavior by default, we wont need different scoping rules, hence making using the repl, in my opinion easier

To clarify

I am suggesting

  1. an explicit global let , which is allowing the use of the key word global with a let block to reverse the scoping of variables, from local by default to global by default
  2. an implicit global let inside file, modules and the repl (to unify the scoping rules and allow local variables inside files and modules)

Since you cannot define a module inside a let block , the two will be a bit different

Sorry, I still don’t understand what problem this addresses. Also, given that

I am not sure I understand the aesthetic argument either.

Aesthetic are completely subjective
And I think it is a nice to have feature, that I believe will not cause any trouble

I think that this code , and being explicit about what is private
(and I think using shorter keyword helps even more)

our let 
    x = 1 
    my y = 2 

    my function addx( a )
        println( x + a)        
    end
    
    function addy( a )
        println(y + a)
    end

end

Looks better, than , where you are explicit about globals

let 
    global x = 1 
    y = 2 

    function addx( a )
        println( x + a)        
    end

    global function addy( a )
        println(y + a)
    end

end

I fully understand, that not all will agree, and while functions want to be global by default, variables want to be local by default, so I think if we have both options (both types of let block) will satisfy all tastes

The side benefit of the top example, is that if it becomes the default behavior inside modules and file, it will also unify scoping rules with the repl

Also another interesting side effect, is you will get local variables in modules and files

you can use the @softscope macro from SoftScope.jl for this as well.

s = 0
@softscope for i in 1:10
    s += i
end

I feel like you aren’t aware of the much too long history these kinds of discussions have already had…

Just a few links (most recent to least recent):

Please be aware that these changes have had just so many hours poured in that frankly, it feels a bit imposing bringing it up again so shortly after the initial release of 1.5.

Also, Julia is (hopefully, finally) not at that stage of development anymore.

2 Likes

The warning prints either way, the error is only thrown when the code actually runs:

Projects $ cat test.jl
i = 0
if rand() < 0.5
  for x in 1:10
    i += x
  end
end
println(i)


Projects $ julia test.jl
┌ Warning: Assignment to `i` in soft scope is ambiguous because a global variable by the same name exists: `i` will be treated as a new local. Disambiguate by using `local i` to suppress this warning or `global i` to assign to the existing global variable.
└ @ /d/Documents/Projects/test.jl:4
0


Projects $ julia test.jl
┌ Warning: Assignment to `i` in soft scope is ambiguous because a global variable by the same name exists: `i` will be treated as a new local. Disambiguate by using `local i` to suppress this warning or `global i` to assign to the existing global variable.
└ @ /d/Documents/Projects/test.jl:4
ERROR: LoadError: UndefVarError: i not defined
Stacktrace:
 [1] top-level scope at /d/Documents/Projects/test.jl:4
 [2] include(::Function, ::Module, ::String) at ./Base.jl:380
 [3] include(::Module, ::String) at ./Base.jl:368
 [4] exec_options(::Base.JLOptions) at ./client.jl:296
 [5] _start() at ./client.jl:506
in expression starting at /d/Documents/Projects/test.jl:2
1 Like

What is the difference between the two executions there? I could not get that.

At runtime, a random value is chosen (rand() < 0.5) and depending on that the for loop executes or not. The point is that it doesn’t matter whether the loop runs or not - there is a warning that’s printed:

┌ Warning: Assignment to `i` in soft scope is ambiguous because a global variable by the same name exists: `i` will be treated as a new local. Disambiguate by using `local i` to suppress this warning or `global i` to assign to the existing global variable.
└ @ /d/Documents/Projects/test.jl:4

You probably just missed it, because Discourse doesn’t have fancy syntax highlighting, but in a terminal (or at least my terminal) the warning is clearly visible because of its yellow color:

The Error is only thrown in the branch where the code actually tries to read from a non-existent i in the for loop.

3 Likes

I completely agree with you
But I still think that having the explicit global let and adding our and my a shorter alternatives for global ad local are nice features to add, for some people it will make their code more natural and simpler to read

Having an implicit global let behavior in the some contexts is a bigger and more controversial change

I’m sorry to be blunt, but in that case you must have skipped the section directly preceding the one you’re quoting, since this case is explained in excruciating detail:

So no, I don’t think anything should change here. Just don’t skip part of the documentation and be surprised that something doesn’t work like you think it should.

1 Like

At this point, I don’t think special alias keywords are worth it to disallow our and my as identifiers (a consequence of your proposal, which would be breaking and thus only be slated for 2.0 at the earliest anyway).

I can only repeat, Julia is not at that stage of development anymore (click for more infos on why).

1 Like

This is talking about copying a part of a function from a file piece by piece into the REPL and have it work, not the other way around. If your student had copied the code from the REPL into a function instead (as advised in the very first section of the performance tips), everything would have worked fine. From my point of view, the current behaviour is (as far as I can tell over the last few years) a global maximum of convenience and ease of teaching.

Put your code into functions, people! Makes stuff easier to test, gives you easy gains in terms of performance, makes your fellow researchers happier, allows more optimizations by the compiler, makes your code less brittle, easier to debug and many more benefits.

2 Likes