Is it possible to know if a function has captured local variables?

General Usage
1 
julia> a = fill(1)
0-dimensional Array{Int64, 0}:
1

julia> function foo(b)
           a[] + b
       end
foo (generic function with 1 method)

julia> foo(1)
2

julia> a[] = 2
2

julia> foo(1)
3

julia> some_detector(foo) # returns `true`

Is it possible to develop a function like some_detector to detect that foo has a captured variable that is outside the scope of the body of foo?

I thought Base.isbitstype would be sufficient, but unfortunately that’s not the case:

julia> function useless_detector(::F) where {F<:Function}
           !isbitstype(F)
       end
useless_detector (generic function with 1 method)

julia> useless_detector(foo)
false

A related discovery I had is that even if I perform a deepcopy on foo, I still cannot isolate a:

julia> foo_dc = deepcopy(foo)
foo (generic function with 1 method)

julia> a[] += 1
3

julia> foo_dc(1) == foo(1)
true
1 Like
2

In general you can’t do that for a function, but maybe for a method. I’m not sure how to do it reliably, but there is some information on captured variables in methods(foo)[1].roots. It’s not documented, so no guarantees about anything.

1 Like
3

Also, I found it concerning that

julia> ismutable(foo)
false

because foo is technically mutable through modifying a, and it’s not like String that is intended to be treated like an immutable object, since closure is fully supported and even documented in Julia’s official documentation.

1 Like
4

That’s not what immutable means.

struct Foo
    x::Array
end

is immutable even though you can modify the array that it points to:

julia> foo = Foo([1,2,3])
Foo([1, 2, 3])

julia> foo.x[1] = 7
7

julia> foo
Foo([7, 2, 3])

Immutability is about what the compiler is allowed to do with a Foo object, not a promise that nothing referenced by foo can change.

It analogous to

const a = [1,2,3]

being const, even though the contents of a can be changed.

5

That’s not capture by a closure, just accessing a global variable, which is a lot harder to figure out from the implementation. It seems like the Method instance’s roots field holds global variables, not sure though. but it holds many other things and I have no idea how to test for attempted global variable accesses when the global variable hasn’t been defined yet.

6

Thanks for the reply. I understand that an immutable composite type instance can hold a mutable field, which I assume is what you are referring to. However, I was unsure if that is what ismutable is trying to do.

Note that your analogy to a global constant variable does not apply to the behavior of ismutable:

julia> const v1 = [1,2,3]
3-element Vector{Int64}:
 1
 2
 3

julia> ismutable(v1)
true

This also holds for the latest Memory type:

julia> const v2 = Memory{Int}([1,2,3])
3-element Memory{Int64}:
 1
 2
 3

julia> ismutable(v2)
true

If for a composite-type instance, ismutable just checks if it is constructed by the keyword struct, that’s fine then.

On a related note, the documentation for immutability of composite types seems misleading/imprecise:

To recap, two essential properties define immutability in Julia:

  • It is not permitted to modify the value of an immutable type.

“value” seems a bit too strong, as for a mutable field in immutable structs, it’s just the reference (pointer) that remains immutable.

7

Thanks for pointing that out! I thought a closure would behave the same as a function capturing a global variable. But it seems not to be the case:

julia> function f2(x::Array{Int, 0})
           f = function (c)
               x[] + c
           end
       end
f2 (generic function with 1 method)

julia> c = fill(1)
0-dimensional Array{Int64, 0}:
1

julia> f2c = f2(c)
#1 (generic function with 1 method)

julia> f2c(1)
2

julia> c[] += 1
2

julia> f2c(1)
3

julia> f2c_dc = deepcopy(f2c)
#1 (generic function with 1 method)

julia> f2c_dc(1)
3

julia> c[] += 1
3

julia> f2c(1)
4

julia> f2c_dc(1)
3

Is there a reason why a global variable captured in a function will not be affected by deepcopy but will be affected when it is passed to a closure?

8

Okay, it seems that it’s not about whether the constructed function is a closure, but about how the global variable is embedded in the function body, which is even more confusing now:

julia> d = fill(1)
0-dimensional Array{Int64, 0}:
1

julia> function f3()
           function (y)
           d[] + y
           end
       end
f3 (generic function with 1 method)

julia> f3c = f3()
#3 (generic function with 1 method)

julia> f3c(1)
2

julia> d[] += 1
2

julia> f3c(1)
3

julia> f3c_dc = deepcopy(f3c)
#3 (generic function with 1 method)

julia> f3c_dc(1)
3

julia> d[] += 1
3

julia> f3c_dc(1)
4

julia> f3c(1)
4
9

That can’t happen. Capturing is just for local variables with scopes that can end. Global variables just exist whereever they are.

1 Like
10

It’s a subtle distinction at first, but a closure is a narrower concept than simply a function referencing a variable from a parent scope. A closure is a function that does this after the parent scope has expired, and therefore needs to carry the variables with it as part of the function object.

This is not a closure:

a = Ref(1)
f(x) = x + a[]

This function doesn’t package anything extra, it’s just a regular function that happens to reference a global variable. Every time this function is called, it will dynamically look up the current value of a in the global scope, according to the normal rules of lexical scoping.

This is a closure:

g = let b = Ref(1)
    x -> x + b[]
end

Here, the lifetime of the variable b will have expired by the time g is assigned, since it was defined in the local scope started by let and ended by end. Thus, it’s up to g to keep b alive such that future calls still work. Therefore, a reference to b is stored as part of the g object. This is what a closure is: a function that comes packaged with one or more bindings from an expired scope. It’s actually just stored as a field. Try it yourself: g.b[] == 1. (This is an implementation detail, though, you shouldn’t rely on it.)

In contrast, there is no f.a. It’s not needed, since a is still alive and well in the global scope, such that f can reference it at any time.

This explains your observations. f2 = deepcopy(f) just gives you a function identical to f. In fact, since functions are immutable singletons, it will return f itself, such that f2 === f. The variable a is not involved, since there’s no reference from the object f to a.

In contrast, g2 = deepcopy(g) will recursively walk the g object, notice the reference to b, and make a copy of b and assemble a new function of the same type as g using the new copy of b. Thus, g.b and g2.b are not the same object, so mutating g.b does not change the behavior of g2, and vice versa.

Hope this helps!

6 Likes
11

Thanks for the explanation!! It’s very clear!!

1 Like
12

!issingletontype will detect many closures, perhaps all although I’m not very sure?

julia> let a = fill(1)
        foo2(b) = a[] + b
        @show typeof(foo2)
        Base.issingletontype(typeof(foo2))
       end
typeof(foo2) = var"#foo2#41"{Array{Int64, 0}}
false

julia> foo3 = let a2 = fill(2)
         foo3(b) = a2[] + b
       end
foo3 (generic function with 1 method)

julia> typeof(foo3)
var"#foo3#42"{Array{Int64, 0}}

julia> Base.issingletontype(typeof(foo3))
false

It does not detect that the original foo depends on global a:

julia> typeof(foo)
typeof(foo) (singleton type of function foo, subtype of Function)

julia> Base.issingletontype(typeof(foo))
true