How does scope work for declaring functions

scope

#1

I have not found any good resources on this. I cannot find anything helpful in the Julia docs entry on scopes. However I have had problems with it and found enough instances were I have no idea what is going on and would be happy if someone could help me:

Example 1:
The code:

fun = 1
for i = 1:3
    fun = i
    print(fun, " ")
end

generates the output
1 2 3
But if I use a function instead:

fun(x) = 1
for i = 1:3
    fun(x) = i
    print(fun(0), " ")
end

It generates the output
1 1 1
What is the difference between a function and a normal variable here? Shouldn’t they work the same?

The next example (closer to my real application) is:

ar = []
for i = 1:3
  eval(:(f() = i))
  println(f())
  push!(ar,f)
end
println(ar[1]()," ", ar[2]()," ", ar[3]()," ",)

this generates

1
2
3
3 3 3

Here I would want to have three different functions in my array, which I can fetch and which yields different values. However the later function declarations seems to overwrite the old ones. Again if I work with a numeric variables it works like expected:

ar = []
for i = 1:3
  eval(:(xx = i))
  println(xx)
  push!(ar,xx)
end
println(ar[1]," ", ar[2]," ", ar[3]," ",)

gives

1
2
3
1 2 3 

Obviously there is something I do not get here, but what?


#2

Try

fun(x) = 1
for i = 1:3
    local fun(x) = i
    print(fun(0), " ")
end

You will find the documentation under local.

But generally, this may not be good style. Consider closures (or an equivalent callable structure). Eg

julia> fs = [() -> i for i in 1:3];

julia> (fs[1](), fs[2](), fs[3]())
(1, 2, 3)

#3

I find very little on closures in the documentation. Is there some place were I can read on what they are? I need it for a part of the DifferentialEquation package were I need to input functions, would closures count as that.

I tried using local and it worked for the first one, but not the second were I used eval. I tried looking closer but now the same code generates an error:

ar = []
for i = 1:3
  eval(:(f() = i))
  println(f())
  push!(ar,f)
end

UndefVarError: i not defined (at the line println(f()))
The code is the same as worked before, but now it seems not to work at all, which makes me very confused. I tried exiting and restarting Juno, but that seems to have no effect.


#4

There is not much more to the documentation of closures than what’s already there.

Don’t use eval. Generally, besides a few corner cases, using eval routinely is a sign that your are doing something that could be done much easier using the standard idioms of the language.


#5

I think in your example above the argument of eval gets evaluated in global scope and that this is always true, hence why it fails. As @tamas_papp said, you should generally avoid using it. It’s really just for code generation. You really don’t want to deliberately invoke the compiler in the middle of your runtime anyway.

We might be able to be more helpful if you give us some idea of what you’re trying to do.


#6

Eval being evaluated on the global scope explains it then. Thank you. I am not sure whenever I can avoid it though, I am building a macro which takes input in a simple format and generates differential equations and other similar structures for me automatically.

I have been able to create the entire function expressions, but then I want to turn those into real function which I can then start doing stuff with. My code reads basically:

func_expressions = []
for i = 1:n
    func_expr = generate_function_expression(data[i])
    eval(:(function func(input); $func_expr end))
    push!(func_expressions,func)
end

the functions in func_expressions are then used as input to an external package.

Is there some way of coverting my expressions into functions without using eval, or am I simply in a very messy part of programming and things are unavoidably difficult?


#7

What you should try to do is to perform any manipulation of expressions that you need in the body of the macro and the plug the results into your output expression. For example

macro diffeq(equation::Expr)
    # here is where you manipulate `equation` and create an expression
    esc(quote
        # here you place the completed expression which will evaluate at runtime
    end)
end

The only exception is cases where you need information generated at runtime in order to know how to manipulate your expression. In that case you have no choice but to invoke the compiler somehow, but you should try hard to avoid this; if you write the appropriate functions it shouldn’t be necessary.

Also, in case you are not aware MacroTools.jl makes writing macro code far less painful (really should have been in Base in my opinion).


#8

Thanks for the help, I will see what I can do.

The MacroTools seems like very useful stuff as well, I will have a look there.


#9

Macros are an advanced concept. Julia is in a strange position of having really powerful metaprogramming facilities, while at the same time these can be difficult to use until you get used to them.

Given that you had problems with closures, I would suggest that you hold off metaprogramming for a while and try to do what you want with functions. Perhaps if you gave an MWE, you could get more specific help.


#10

Even once you are comfortable writing macros, usually a good rule of thumb is to get everything that the macro relies on fully designed and working before you start writing the macro at all. I’ve done a fair amount of meta-programming now and I still find it can be quite a pain in the ass, even though the results are spectacular.


#11

Thanks for the advice. I think I’ve gotten the most important functionalities working now, but it is indeed really messy.


#12

I’m not 100 % sure but I think the observed effects in this snippet have more to do with redefining methods and ‘world-age’ than scope. The last line (added by me) gives 3 as output; and this were not possible if the global function fun wouldn’t have been modified. - To hopefully understand what happens I used the following code:

println("WORLD_COUNTER at start: $(ccall(:jl_get_world_counter, UInt,()))")
if isdefined(:foo)
    println("normal/latest foo(10) as defined: $(foo(10))/$(Base.invokelatest(foo, 10))")
else
    println("foo not defined")
end
foo(x) = (999, x)
println("\nWORLD_COUNTER after foo (re)definition: $(ccall(:jl_get_world_counter, UInt,()))")
println("normal/latest foo(20) after (re)definition: $(foo(20))/$(Base.invokelatest(foo, 20))\n")
for i = 1:3
    # local foo(x) = (i, x)  # def./upd. foo; after loop exit, global foo(x) = (99,x) is visible again 
    foo(x) = (i, x) # only update latest defined foo; after exit, foo(x) = (3,x) remains
    println("    WORLD_COUNTER after forloop foo redefinition: $(ccall(:jl_get_world_counter, UInt,()))")
    println("    normal/latest foo(30+$i) within forloop: $(foo(30 + i))/$(Base.invokelatest(foo, 30 + i))")
end
println("\nWORLD_COUNTER after for block: $(ccall(:jl_get_world_counter, UInt,()))")
println("normal/latest foo(40) after for block: $(foo(40))/$(Base.invokelatest(foo, 40))")
(--- output)

WORLD_COUNTER at start: 22146                                                                                                                
foo not defined                                                                                                                              
                                                                                                                                             
WORLD_COUNTER after foo (re)definition: 22147                                                                                                
normal/latest foo(20) after (re)definition: (999, 20)/(999, 20)                                                                              
                                                                                                                                             
    WORLD_COUNTER after forloop foo redefinition: 22148                                                                                       
    normal/latest foo(30+1) within forloop: (999, 31)/(1, 31)                                                                                        
    WORLD_COUNTER after forloop foo redefinition: 22149                                                                                       
    normal/latest foo(30+2) within forloop: (999, 32)/(2, 32)                                                                                        
    WORLD_COUNTER after forloop foo redefinition: 22150                                                                                       
    normal/latest foo(30+3) within forloop: (999, 33)/(3, 33)                                                                                        
                                                                                                                                             
WORLD_COUNTER after for block: 22150                                                                                                         
normal/latest foo(40) after for block: (3, 40)/(3, 40) 

The output of the ‘normal’ and the ‘latest defined’ foo call shows that the ‘latest foo’ does get updated in the for loop. I suppose that the loop simply cannot replace a method received from the outside. The method gets replaced after the loop though. When you write local foo(x) = i, then there is a locally defined foo method which can be replaced in every loop iteration. After the loop, the global foo method is visible again.


#13

Thank you swissr, that is really helpful. I did not know about that at all before.