Best solution to Julia's soft scope problem?

General Usage
1

For example, a loop like the below will consistently cause an error.

A = 5
for i in 1:10
    A = A+i;
end

Error message:

Assignment to `A` in soft scope is ambiguous because a global variable by the same name exists: `A` will be treated as a new local. Disambiguate by using `local A` to suppress this warning or `global A` to assign to the existing global variable.
**ERROR:** LoadError: UndefVarError: A not defined
Stacktrace:
[1] top-level scope

Other than wrapping everything up with a let and end, what is the best solution to this problem? It seems that they are not interested in fixing this in Julia 2.0?

2 Likes
2

Your post is a bit ambiguous about what you’re doing (it would help to be more specific), because if you just copy your code into a REPL it works fine on Julia 1.5 and higher AND it works inside a function. The only time you get that error is if you include it in a script (but I had to guess that’s what you were doing).

The thing to understand is that Julia is trying to coach you to better programming practice. Don’t work in global scope except when you’re playing around in the REPL. Many Matlabbers fall victim to this practice and it holds back their growth as good programmers. Get used to creating functions, think about design, testability, etc., and you’ll get better performance and cleaner code.

19 Likes
3

Thanks, Tim. Yes, the only time it gets me the error is when I include it in a script.

The problem of wrapping them into a function is that sometimes, I have dozens of input variables. The example is a much simplified one.

1 Like
4

A solution is to create a struct that hold all of those variables, so that functions can have just one argument that makes all of the variables available.

7 Likes
5

All the more reason to use functions: loops at global scope have terrible performance, but loops in a function have amazing performance. The more data you’re working with, the more this will matter.

But it’s also worth asking if those dozen arguments might lead a more productive life with a certain amount of organization into structs, Dicts, etc.

This past Wednesday I taught a lecture that describes how thinking about testability can help clarify better ways of approaching code design: https://github.com/timholy/AdvancedScientificComputing/blob/main/schedule/schedule_2021.md (see the Oct. 20 session).

18 Likes
6

Even for smaller tasks, here is a tip: wrap everything in a function:

function job()

end

Use Revise and include the script with:

julia> using Revise

julia> includet("./job.jl") # note the t for track

julia> job()

Now modify your script. You only need to run job() again to see the changes in action.

This is very practical even of you are only tuning the appearance of a plot, for example.

(Kudos to Tim)

13 Likes
7

This is a compromise between being technically correct and just doing it. This creates a local scope for you to work in.

let A = 5
    for i in 1:10
        A = A+i;
    end
    println(A)
    # Do other stuff that uses A
end

If you really want want the minimum fix, you can add a global declaration in the loop. I do not recommend it.

A = 5
for i in 1:10
    global A = A + 1
end
10 Likes
8

Tim, Many thanks for sharing the course material! :+1: I definitely want to take advantage of it.

9

I have been programing with matlab exclusively for over 15 years. I seldom use functions in matlab. The problem is that, with functions, many variables are not visible in the workspace. That makes the debugging harder.

10

Of course, if you have a long experience working in a different language, learning to program in a new one (be it Julia or anyone else) means modifying the workflows you are accustomed to. Working with functions (which in Matlab usually has the annoying cost of creating a file for each function) should be one of the first, if you want to work smoothly in Julia.

There are other tips for Matlab users here:
https://docs.julialang.org/en/v1/manual/noteworthy-differences/#Noteworthy-differences-from-MATLAB

With respect to the difficulty of debugging code if it is in functions, again it depends on your debugging workflows. There are several tools in (Debugger.jl, Infiltrator.jl and the built-in debugger of VS Code, among others) to facilitate debugging in Julia, which are actually meant to be used on functions.

N.B.: If you want to continue a discussion on debugging workflows and tools for Julia, I advise you to:

  1. Search for earlier discussions about that topic in this forum and read them. There have been many, and many of them long. You shouldn’t like to start over again a heated discussion on a worn topic.
  2. If there is any new point that you think deserves further discussion, start a new thread, instead of posting your comments in this old one.
11 Likes
11

I almost always debug function code by just copying it into a REPL in global scope. The pattern is simple: write functions, debug them in global scope. Don’t write code that works in global scope. That is slow and unnecessary.

4 Likes
12

Generally, making most of your variables persistent is strongly discouraged because your memory gets used up and harder to manage when you go beyond small scripts. MATLAB’s particular separation of base (script variables), global (global variables), and function workspaces helps make this workflow easier, but the poor scalability remains. Good performance is maintained by calling often highly optimized C/C++/Fortran and JIT compilation (JIT is claimed to speed up customer workflows by ~2.23x on average).

Like other languages, MATLAB performance tips encourage functions and discourage global variables among other things. Not only do the performance tips allow you to scale further without persistent variables sabotaging you, but they let the JIT compiler optimize better. MATLAB provides a debugger that can handle functions, much like those in other languages. However, the tradeoffs of the workspaces does hamper this style (not as much now), and users tend to stick with simpler suboptimal practices.

That’s not necessarily a bad thing, “suboptimal” can also mean “more than good enough,” especially with how MATLAB’s implementation helps. However, MATLAB users tend to experience a particular culture shock when going to languages that didn’t make the same tradeoffs to mitigate functionless code. It’s normal to feel frustration at this; my first language was also MATLAB, I know how it feels to suddenly have to care about functions and modules all the time. But if MATLAB doesn’t even endorse functionless code as best practice, we can’t really expect other languages to.

Like MATLAB, Julia’s JIT compiler optimizes functions better, but the difference is often MUCH larger than ~2.23x, hence the particularly strong emphasis. Even when making some necessary persistent variables in the global scope, the heavier computation is typically done by a function call.

7 Likes
13

Matlab has a wonderful debugger. Debugging inside a function is as simple as put the breakpoint inside that function and when it stops one has easy access to all variables in the viewer.

The Julia debugger tries to provide the same behavior but the usability falls far behind that of Matlab. But that’s what we have and it is already very useful.

10 Likes
14

This thread is a very disheartening read for someone trying to use Julia’s powerful libraries for quick experimentation with a new idea. I can’t fathom why Julia needs to enforce scope to “coach [us] to better programming practice”. Who cares if my code is slow, besides me? I just want to try running a for loop. For what it’s worth I am trying to debug something like the below and getting scope errors out the kazoo:

nIter = 10
numReals = zeros()
p_star = zeros(27)
currNumReals = 0
for i = 1:nIter
    new_model = MvNormal(p_star,I)
    param_star = rand(new_model,1)
    ...
    a = solve(F)
    tempNumReals = length(real_solutions(a))
    if tempNumReals > currNumReals 
        p_star = vec(param_star)
        currNumReals = tempNumReals
    else
        p_star = p_star
        currNumReals = currNumReals
    end
    numReals[i] = currNumReals
end
2 Likes
15

Sorry you’re getting frustrated. Welcome!

The manual is great on this topic! It has been carefully written because so many new users get confused.
https://docs.julialang.org/en/v1/manual/variables-and-scoping/#scope-of-variables

You haven’t included all the code or the error messages. It would also help to know if this code is run in the REPL, a function, or in a standalone script because then we would know, for example if nIter is a global or local.

Basically, if its as a script or in a bare model, then the outer functions are globals. Attempts to assign to them inside a loop will create a new local variable with the same name that is completely distinct. The logic is that functions and all that are global variables. The global namespace is crowded with important variables, and you want to be able to use f as variable name in a loop without thinking or checking whether its already a function name.

Just as an aside, this old thread might not be the best place for your question. The title is a little misleading: There is no soft scope “problem” with Julia, only some decisions have been made about scope that make many things easier and a few things counterintuitive. It takes practice to reason with these rules and they may be different in your “native programming language”.

5 Likes
16

I see three ways to help alleviate your scope problem :

First is to add at beginning the global déclaration relative to variables used and updated inside the loop (easier way, but costly in speed if you don’t const your variables)

global nIter, numReals, p_star, currNumReals

Don’t forget to change numReals = zeros() to numReals = zeros(nIter) so that the numReals[i]= … is not out of bounds.

The two others solutions keep the speed, but are more difficult for debugging( see also at end)

Second (preferred? best in style?) is to transform to a function :

nIter = 10
function compute(nIter)
  numReals = zeros(nIter)
  p_star = zeros(27)
  currNumReals = 0
  for i = 1:nIter
    new_model = MvNormal(p_star,I)
    param_star = rand(new_model,1)
    ...
    a = solve(F)
    tempNumReals = length(real_solutions(a))
    if tempNumReals > currNumReals 
        p_star = vec(param_star)
        currNumReals = tempNumReals
    else
        p_star = p_star
        currNumReals = currNumReals
    end
    numReals[i] = currNumReals
  end
  return numReals
end
numReals=compute()

Third is to use a let block :

nIter = 10
numReals=let nIter=nIter
  numReals = zeros(nIter)
  p_star = zeros(27)
  currNumReals = 0
  for i = 1:nIter
    new_model = MvNormal(p_star,I)
    param_star = rand(new_model,1)
    ...
    a = solve(F)
    tempNumReals = length(real_solutions(a))
    if tempNumReals > currNumReals 
        p_star = vec(param_star)
        currNumReals = tempNumReals
    else
        p_star = p_star
        currNumReals = currNumReals
    end
    numReals[i] = currNumReals
  end
  numReals
end

As was said earlier you could use Infiltrator inside the function or the let block for debugging.

Hope that helps,

2 Likes