Generate a list of functions

Hello everyone!
I want to generate a list of functions, and run into some unexpected behaviour.

# Case 1: using anonymous functions
fs=[]
for i in 1:3
    f=()->i
    push!(fs,f)
end
@show [p() for p in fs]

# Case 2: create function with a new name
gs=[]
for i in 1:3
    function g()
        i
    end
    push!(gs,g)
end
@show [p() for p in gs]

# Case 3: create function with a used name
h()=100
hs=[]
for i in 1:3
    function h()
        i
    end
    push!(hs,h)
end
@show [p() for p in hs]

The output is like

[p() for p = fs] = [1, 2, 3]
[p() for p = gs] = [1, 2, 3]
[p() for p = hs] = [3, 3, 3]

Apparently, the behaviour of Julia in case 3 is different from that in case 2. Although the definition of h() is my sample code is fairly close to the loop, in real program, it can be in pretty far ahead, or in other files, which may cause some issues.

Also, we tell from the output, the h() function is neither the expected ones as in case 2, nor the one defined outside the loop, which should return 100.

I am not sure whether this behaviour is documented somewhere or not. If it is already documented, a warning should be given here.

I tested code above using Julia 1.7.1 on Ubuntu 20.04.

With best wishes,
Yu

I think the for-loop scope rule is messing with you. When you have h() from before the for loop, and use (by push!()ing it) it inside the loop, somehow Julia decides to evaluate the function definition in global scope.


julia> g()
ERROR: UndefVarError: g not defined
Stacktrace:

julia> h()
3

Thank you for the reply.
In such case, I would expect three 100 output, if the global h is used, as I defined h()=100.

why? I mean, it’s the same principle as:

julia> gs=[]
Any[]

julia> for i in 1:3
           g = i
           push!(gs,g)
       end
julia> g
ERROR: UndefVarError: g not defined

julia> gs
3-element Vector{Any}:
 1
 2
 3

julia> h = 100
100

julia> hs = []
Any[]

julia> for i in 1:3
           h = i
           push!(hs,h)
       end

julia> hs
3-element Vector{Any}:
 1
 2
 3

julia> h
3

except the global h() function is evaluated later in [h() for h in hs]

I remove the misleading [h() for h in hs] and replace it as [p() for p in hs]. The problem is not located in the @show lines.

The non-function analogy is like

x=100
xs=[]
for i in 1:3
    x=i
    push!(xs,x)
end
@show [p for p in xs]
@show x

and the output is

[p for p = xs] = [1, 2, 3]
x = 3

which is reasonable, because the global x is rebound to the value of for-loop local i, and after the loop, x==3.

While in my “list of function” sample, if the rule in non-function case hold, I would expect the function in the list returns [1, 2, 3], or the global function f() is used, returns [100, 100, 100]. But none of these happens. Instead, the final answer is [3, 3, 3], it seems the i is stuck to the last iteration of the loop, while it is not in case 2.

a better analogy may be this?

julia> hs = []
Any[]

julia> h = [100]
1-element Vector{Int64}:
 100

julia> for i in 1:3
           h[] = i
           push!(hs,h)
       end

julia> hs
3-element Vector{Any}:
 [3]
 [3]
 [3]

you’re calling the function h() only after the list is constructed. You can only have one h() (a named function) in the Main scope, so when you evaluate it after the loop, by construction you would get the same value.

Becuase h existed before the loop and you use h on the RHS in the loop, it all refers to the same h – the named function h() in Main scope.

2 Likes

or, if you evaluate the h() inside the push(), it will reproduce the example when h=i is just an immutabe scalar:

julia> h()=100
h (generic function with 1 method)

julia> hs=[]
Any[]

julia> for i in 1:3
           function h()
               i
           end
           push!(hs,h())
       end

julia> hs
3-element Vector{Any}:
 1
 2
 3

basically you re-define the Main.h() in each iteration, and call Main.h() each iteration as well, instead of delaying the evaluation after the loop.

Got the idea. Main.h is modified, and all three element in the hs refers to it.

1 Like

Although somehow I understand this analogy… Is that means a function is mutable?
When I am ready to accept this, I find

f()=1
@show ismutable(f)

gives false.

I think this may be a semantics problem. ismutable probably expects an instance of a struct or something like that. The ismutable documentation says:

ismutable(v) → Bool

Return true iff value v is mutable. See Mutable Composite Types for a discussion of immutability. Note that this function works on values, so if you give it a type, it will tell you that a value of DataType is mutable.

Functions are mutable in the sense new methods can be added to them, and their previous methods can be redefined.

If I’m interpreting the documentation correctly this assignment case is ambiguous and will give different results when evaluated in interactive vs. non-interactive cases:

''When x = <value> occurs in a local scope, Julia applies the following rules to decide what the expression means based on where the assignment expression occurs and what x already refers to at that location:

  1. Existing local: If x is already a local variable , then the existing local x is assigned;
  2. Hard scope: If x is not already a local variable and assignment occurs inside of any hard scope construct (i.e. within a let block, function or macro body, comprehension, or generator), a new local named x is created in the scope of the assignment;
  3. Soft scope: If x is not already a local variable and all of the scope constructs containing the assignment are soft scopes (loops, try / catch blocks, or struct blocks), the behavior depends on whether the global variable x is defined:**
    ** * if global x is undefined , a new local named x is created in the scope of the assignment;**
    ** * if global x is defined , the assignment is considered ambiguous:
    ** * in non-interactive contexts (files, eval), an ambiguity warning is printed and a new local is created;**
    ** * in interactive contexts (REPL, notebooks), the global variable x is assigned.**

You may note that in non-interactive contexts the hard and soft scope behaviors are identical except that a warning is printed when an implicitly local variable (i.e. not declared with local x ) shadows a global. In interactive contexts, the rules follow a more complex heuristic for the sake of convenience. This is covered in depth in examples that follow.’’

I created a file containing the code:

module App

h()=100
hs=[]
for i in 1:3
    function h()
        i
    end
    push!(hs,h)
end
@show [p() for p in hs]
println("global h function $(h())")

end # module

and then did a non-interactive evaluation

julia> using App
[ Info: Precompiling App [4c3466e5-b909-4ae6-bfb7-8e1039e25e36]
[p() for p = hs] = [1, 2, 3]
global h function 100

In this case a new function h() was bound each time through the loop and the global h() was unaffected as @zhaiyusci expected.

1 Like

Great! You reminded me that my concern is related to a scope issue.
I forgot to test my example in a non-interactive environment, but only in a Jupyter notebook.
Interestingly, the soft-local-scope warning is not raised if a function in local scope is redefined, as in Julia 1.7. I am not sure whether it is a pitfall in JuliaLang. But a warning will help.