Global variable issue

[jiang_ming_zhang]

It is quite strange that with the following code

N = 10 
M = 10 
dim = binomial(N+M-1, M-1)
basis=zeros(Int8, M, dim)
basis[1,1]=N
s=1
while basis[M,s]<N 
    s1=M-1
    while basis[s1,s]==0
        s1=s1-1
    end
    
    basis[1:s1-1,s+1]=basis[1:s1-1,s]
    basis[s1,s+1]=basis[s1,s]-1
    basis[s1+1,s+1]=N-sum(basis[1:s1,s+1])
     
    s=s+1
end

I got the error

┌ Warning: Assignment to `s` in soft scope is ambiguous because a global variable by the same name exists: `s` will be treated as a new local. Disambiguate by using 
`local s` to suppress this warning or `global s` to assign to the existing global variable.
└ @ Untitled-2:17
ERROR: UndefVarError: `s` not defined
Stacktrace:
 [1] top-level scope
   @ .\Untitled-2:9

The global variable ‘s’? where is it? I even restarted my vscode, but the error still pops up.

[nsajko]

The error message is quite good. It points to the global s, pointing out the ambiguity in the line s=s+1.

@jiang_ming_zhang

The global variable ‘s’? where is it?

Here:

@jiang_ming_zhang

s=1

[jiang_ming_zhang]

but what is wrong with this line?

s = 1 

is such a common initiation of a counting variable. The same code works perfectly with matlab.

[Benny]

@jiang_ming_zhang

The same code works perfectly with matlab.

Unlike Octave, Julia is a fundamentally different language from MATLAB, so don’t expect the most direct code translations to behave the same. In this case, a global variable must be declared global in local scopes in source files. In the REPL this requirement is relaxed for some local scopes like loops for convenience, that is why the warning talks about “soft scopes.” The warning suggests this fix, which resolves the error:

while basis[M,s]<N 
   global s

Global variables are not easy to reassign from local scopes in scripts by design. You don’t want to reassign a global variable when you accidentally reuse a name in a local scope. A global scope can be split across many files, so that global variable may be in another file out of your text editor’s sight.

[heliosdrm]

For a better understanding of what is happening and why, I recommend reading the section of the Julia manual on scopes:
https://docs.julialang.org/en/v1/manual/variables-and-scoping/

[nsajko]

@jiang_ming_zhang

The same code works perfectly with matlab.

So you thought Julia is a reimplementation of Matlab? What gave you that impression?

[nsajko]

@jiang_ming_zhang

but what is wrong with this line?

No one said there’s anything wrong with it…

[heliosdrm]

Expanding on @Benny’s answer, which gives the solution to tell that the global variable s is the “counting variable” used inside the loop:

If the code you posted was part of the body of a function, the loop would work as you expected, for the reasons explained in the section of the manual linked above.

Generally, in Julia it is very recommendable to encapsulate as much code as possible in functions, and reduce the code executed in the global scope of scripts to a few lines loading packages and calling those functions. But this particular issue is the least of good reasons to do that: it is also important to make the code run faster and easier to debug.

[mkitti]

Here’s another fix using let to create a local scope.

let

N = 10
M = 10
dim = binomial(N+M-1, M-1)
basis=zeros(Int8, M, dim)
basis[1,1]=N
s=1
while basis[M,s]<N
    s1=M-1
    while basis[s1,s]==0
        s1=s1-1
    end

    basis[1:s1-1,s+1]=basis[1:s1-1,s]
    basis[s1,s+1]=basis[s1,s]-1
    basis[s1+1,s+1]=N-sum(basis[1:s1,s+1])

    s=s+1
end

end # let

You could also pack it into a function.

function foo()

N = 10
M = 10
dim = binomial(N+M-1, M-1)
basis=zeros(Int8, M, dim)
basis[1,1]=N
s=1
while basis[M,s]<N
    s1=M-1
    while basis[s1,s]==0
        s1=s1-1
    end

    basis[1:s1-1,s+1]=basis[1:s1-1,s]
    basis[s1,s+1]=basis[s1,s]-1
    basis[s1+1,s+1]=N-sum(basis[1:s1,s+1])

    s=s+1
end

return basis

end
foo()

Or as others have recommended declare that you want to use a global within the loop.

N = 10
M = 10
dim = binomial(N+M-1, M-1)
basis=zeros(Int8, M, dim)
basis[1,1]=N
s=1
while basis[M,s]<N
    s1=M-1
    while basis[s1,s]==0
        s1=s1-1
    end

    basis[1:s1-1,s+1]=basis[1:s1-1,s]
    basis[s1,s+1]=basis[s1,s]-1
    basis[s1+1,s+1]=N-sum(basis[1:s1,s+1])

    global s=s+1
end

[A_C]

I too find it frustrating too use global for simple scripts. I had read about a workaround on discourse but cannot find it now. I had to figure it using the SoftGlobalScope.jl
package.

Suppose you have a Julia file named tryscope.jl

s = 0.0
for i = 1:10
    s = s+1
end
println(s)

In order to run this successfully in soft global scope type the following in Julia REPL.

julia> using REPL

julia> Base.include(REPL.softscope, Main, "tryscope.jl")
10.0

Disclaimer: I do not know the full consequences of doing this.

[jiang_ming_zhang]

why is ‘basis’ not treated as a global variable? It is also defined outside of the for loop.

[A_C]

This old thread might be helpful.

[lmiq]

If you encapsulate everything into a function, you can use Revise and that gives you a very convenient workflow in Julia:

file test.jl, containing:

function compute_basis(;N=10, M=10)
  dim = binomial(N+M-1, M-1)
  basis=zeros(Int8, M, dim)
  basis[1,1]=N
  s=1
  while basis[M,s]<N
      s1=M-1
      while basis[s1,s]==0
          s1=s1-1
      end
      basis[1:s1-1,s+1]=basis[1:s1-1,s]
      basis[s1,s+1]=basis[s1,s]-1
      basis[s1+1,s+1]=N-sum(basis[1:s1,s+1])
      s=s+1
  end
  return basis
end

in the REPL, do:

julia> using Revise

julia> includet("./test.jl") # NOTE de t after include!

julia> compute_basis()
92378

julia> compute_basis(N=5)
2002

And if you modify the function in the file and save the file, the changes will automatically be updated the next time you run it.

I have notes about both things here: Scope of loops · JuliaNotes.jl and Development workflow · JuliaNotes.jl

So, for example, I run:

julia> using BenchmarkTools

julia> @btime compute_basis()
  7.851 ms (184756 allocations: 12.16 MiB)
92378

Now I modified the function to improve its performance, and just ran it again:

julia> @btime compute_basis()
  2.528 ms (2 allocations: 902.23 KiB)
92378

The changes were:

• Add @views before function, because slices in Julia allocate new arrays.
• Add a dot . here, and then you are performing a element-by-element substitution:
  basis[1:s1-1,s+1] .= basis[1:s1-1,s]

modified code

@views function compute_basis(;N=10, M=10)
  dim = binomial(N+M-1, M-1)
  basis=zeros(Int8, M, dim)
  basis[1,1]=N
  s=1
  while basis[M,s]<N
      s1=M-1
      while basis[s1,s]==0
          s1=s1-1
      end
      basis[1:s1-1,s+1] .= basis[1:s1-1,s]
      basis[s1,s+1]=basis[s1,s]-1
      basis[s1+1,s+1]=N-sum(basis[1:s1,s+1])
      s=s+1
  end
  return basis
end

[stevengj]

@A_C

I too find it frustrating too use global for simple scripts.

(Julia is really heavily oriented around putting all non-trivial code into functions. It’s best not to fight the language on this.)

[Nathan_Boyer]

basis, s, N, and everything else defined outside the loop are global variables, meaning they are defined at the topmost level. The line s=s+1 is inside the local scope of the while loop. Here Julia is not sure if you want to use the global s or define a new local s variable, so it throws an error. There are local and global keywords that can force one behavior or the other.

This issue is tricky to work around at first, especially with while loops when testing and learning the language (with scripts), but it will go away once you start using Julia more like how it was designed to be used (with functions that accept global variables as arguments). I went through the same transition and had many of the same struggles you are facing. @mkitti’s answer has the common workarounds. You can use let or global as quick fixes for your scripts, but the long-term solution is to starting thinking in terms of functions. If you use functions, then you don’t need to bother learning the complicated hard/soft scoping rules; the loops will work as you expect.

Some links:

Memory Pre-allocation in the global scope

Let me try to summarize (because there is a lot of confusion around this): Your initial approach is fine in terms of performance. “Avoid global variables” means avoid inheriting global variables through scoping. Your code will use global variables as inputs and outputs; this is normal. F1!() could have been defined without taking any arguments and still work. Then it would have to obtain the values from the outer (in this case global) scope. That would be bad practice and slow unless those …

Why does the soft scope warning not appear in a nested loop?

You can always declare a variable as local to avoid it picking up any previous definitions: function f() i = 100 for j in 1:2 local i = 0 i = i + j end println(i) end julia> f() 100

Why and when to pass variables to sub-functions?

I am working on spitting my functions up into many smaller sub-functions. The issue I came across is that I need to pass more and more variables to successive sub-functions. How is this typically handled? I see three possibilities: Pass all the needed variables to every sub-function. Don’t pass any variables to sub-functions and rely on inheritance from outer scope. Re-compute quick one-liners like array sizes inside every sub-function and then only pass the arrays themselves. Option 1 seems…

[A_C]

I should not have used “frustrating”. It is just mildly inconvenient for smaller scripts. I do use functions for codes which are to be maintained for longer time. However, in initial phases when one is doing exploratory coding it is more convenient to have REPL like scoping rules.
One of the disadvantages of using functions or let blocks is that you lose access to variable explorer which is very convenient for debugging (especially for matrices). One can recover the convenience to some extent by using Debugger or Infiltrator. But matrices are not as nicely displayed as in variable explorer.
One question that I had was are there any downsides to use Base.include(REPL.softscope, Main, "tryscope.jl") for smaller scripts?

[heliosdrm]

Your comment on the variable explorer suggests that you might be using some tool like VS Code. And in that editor at least, there are convenient shortcuts for sending blocks of code to the REPL, without include-ing files. That is the way I try exploratory coding in initial phases, and that keeps the “soft scope” behavior of the REPL.

[stevengj]

@A_C

However, in initial phases when one is doing exploratory coding it is more convenient to have REPL like scoping rules.

I usually do that in interactive environments like Jupyter notebooks, which do have REPL-like scoping rules.

[A_C]

This is good to know. Thanks. So if I select the whole file and press ctrl+enter it will execute with REPL-like scoping. It is mentioned in the VS Code docs:

For most users, this should be their default command to run Julia code in the REPL

I thought hitting the run button was the default.

Nowadays, I start REPL from shell in VS Code and use infiltrator (sometimes Debugger).

[Benny]

@A_C

One question that I had was are there any downsides to use Base.include(REPL.softscope, Main, "tryscope.jl") for smaller scripts?

It uses base Julia so it should work as well as pasting code into a begin block in the REPL, which has its own downsides as mentioned already. Personally I wish the REPL didn’t default to soft scope rules and just offered this as a manual option, it legitimately is confusing for scoping rules and warnings to contend with 2 default behaviors, even across file types.

@jiang_ming_zhang

why is ‘basis’ not treated as a global variable? It is also defined outside of the for loop.

It is. You just never reassign it in the loop, so the local scope accesses the global. The way the scoping rules are set up, a scope is free to access variables from outer scopes it does not reassign, even if it mutates the assigned instance. Earlier I mentioned that reassigning global variables in local scopes are not allowed by default to prevent assignments of accidentally identical names in local scopes from unintentionally changing and accessing global variables. That’s not a problem for mutation without assignment because it’s unambiguous you must involve an outer scope’s variable.