Surprising capture boxing behavior in closure

Specific Domains Julia at Scale
#1

Hi, I was surprised by the boxing behavior of closure capture, and I’d love to hear about its motivation and discuss whether it’s still relevant.

I came across this when I noticed how it can introduce a race condition in @async calls. I opened an issue here to discuss that:

I didn’t know about boxing during closure capture before encountering this code.

Here is a simple example. I would’ve expected this to print 0, but it prints 1:

julia> function closure_surprise()
           x = 0
           # I would've expected the closure to be "constructed" here, with the value in `x`
           # captured by copy at _this point_.
           f = ()->println("INNER: $x")
       
           x = 1
           f()
       end
closure_surprise (generic function with 1 method)

julia> closure_surprise()
INNER: 1

Against my expectations, when the outer-scope modifies x, it also changes the value of x seen inside the closure.

Let me expand a bit on why this wasn’t what I expected:

  1. The docs imply x should be a copy:
    https://docs.julialang.org/en/v1.1.0/devdocs/functions/#Closures-1

    function adder(x)
    return y->x+y
end

    is lowered to (roughly):

    struct ##1{T}
    x::T
end

(_::##1)(y) = _.x + y

function adder(x)
    return ##1(x)
end

    If that’s really how it was implemented, _.x would simply be an Int, whose value would be set to whatever x was at construction time.

  2. You can return an anonymous function that captures local variables without wondering what happens when the local variable goes out of scope. And that’s because it’s copied into closure, not passed as a reference to a local stack-allocated variable.

  3. This doesn’t match up with the behavior in the rest of Julia for immutable types like Int: a variable name is supposed to just be a name that refers to a value. Assigning x=0; y=x doesn’t bind y to x; instead they both point at x's current value, and changing x doesn’t change y.
    Even an actual reference (Ref) to an integer variable just becomes a reference to a copy of the variable!

    julia> begin
         x = 0
         xref = Ref(x)
         x = 1
         xref[]
       end
0

  4. Lambdas are pass-by-copy by default in C++, which might be part of why I expected that here.

That said, I now understand that this is following the rules referenced here in the Performance Tips:

This style of code presents performance challenges for the language. The parser, when translating it into lower-level instructions, substantially reorganizes the above code by extracting the inner function to a separate code block. “Captured” variables such as r that are shared by inner functions and their enclosing scope are also extracted into a heap-allocated “box” accessible to both inner and outer functions because the language specifies that r in the inner scope must be identical to r in the outer scope even after the outer scope (or another inner function) modifies r .

And indeed, we can see that is what is happening:

julia> @code_lowered closure_surprise()
CodeInfo(
2 1 ─      x = (Core.Box)()
  │        (Core.setfield!)(x, :contents, 0)
5 │        #76 = %new(Main.:(##76#77), x)
  │        f = #76
7 │        (Core.setfield!)(x, :contents, 1)
8 │   %6 = (f)()
  └──      return %6
)

And we can also see that if we delete the x=1 at the end of the function, x stops being Boxed:

julia> @code_lowered closure_surprise()
CodeInfo(
80 1 ─      x = 0
83 │   %2 = Main.:(##84#85)
   │   %3 = (Core.typeof)(x)
   │   %4 = (Core.apply_type)(%2, %3)
   │        #84 = %new(%4, x)
   │        f = #84
86 │   %7 = (f)()
   └──      return %7
)

The problem I have here, is that I can’t figure out the right way to get the behavior I wanted through closure. Inside the function, I can’t mark it local or anything, and it’s too late to make a copy.

Okay, so all that said, I’m sure there are interesting reasons that I don’t yet understand for why this is done. Are there situations where the boxing is necessary? I’d love to hear those if anyone can spare the time! :slight_smile: Thanks!

3 Likes
Function closures and ifelse with unexpected behaviour
#2

On the other hand, the doc about scoping and let uses closure creation as an example illustrating the use of let to avoid two closures sharing the same variable.

Wouldn’t using let be the right way to get what you want here too?

function closure_surprise()
    x = 0

    f = let x = x
        ()->println("INNER: $x")
    end
    
    x = 1
    f()
end

julia> closure_surprise()
INNER: 0
1 Like
#3

A while ago there was some discussion that “freezing the captures” might make sense in FastClosures https://github.com/c42f/FastClosures.jl/issues/5. I didn’t get around to implementing that but most of the machinery is there. It also probably makes more sense than what I currently have in that package :wink:

I’ve wondered about the exact reason for this as well. It seems like it’s been there from the very first closure lowering in https://github.com/JuliaLang/julia/commit/6ed5f4e89a995a4f0ff968a76f4906a865d4e334. It certainly makes closures with mutable state easy to construct but that seems a little at odds with modern julia preferring immutable structs, etc. It would be quite easy to explicitly use a Ref for occasions which require mutable state. @jeff.bezanson are there points of interest that we’re missing here?

#4

Always make passed in values const and require Ref for mutation would be my preference too (https://github.com/JuliaLang/julia/issues/14959#issuecomment-228044396). That could perhaps avoid having to Box?

4 Likes
#5

I tend to agree with you. Coming from C++ I find the scheme-alike lexical scoping rules quite counter intuitive. Perhaps because of a dissonance between imperative vs functional styles. However, any alternative would seem to need a story about how mutually recursive inner functions would refer to each other.

It’s interesting to re-read the discussion at https://github.com/JuliaLang/julia/issues/16727.

We can at least have a macro which introduces independent bindings. FastClosures doesn’t actually do this (mainly because I didn’t fully understand the rules when I wrote it). But that could be fixed; see https://github.com/c42f/FastClosures.jl/issues/11.

1 Like
#6

Ref is type stable, but sometimes people might need to use a dynamically typed free variable, and its type could change. So totally using Ref instead of Core.Box might not be a good idea.

However, when we can assure the free variable is stably typed, we should use Ref, which is not implemented in Julia.

2 Likes
#7

I think that

julia> C = Ref{Any}(0)
Base.RefValue{Any}(0)

julia> C[] = "something"
"something"

is an option, too.

4 Likes
#8

I just want to follow up and note that a workaround for this behavior during concurrent programming was introduced in v1.4, here:

As of 1.4, you can use $ to interpolate an argument into a @spawn or @async call, which will prevent this surprise boxing. For example, this code is protected against the surprise shared reference, and will work like we intended:

function countup()
    i = 0
    while i < 5
        @async println($i)
        i += 1
    end
end

julia> countup()
0
2
1
3
4

It only works for @async and @spawn, not for arbitrary lambdas, but I think it’s helpful, and you can follow the same pattern using let for arbitrary lambdas.

#9

We also talked about the performance implications of this again today at JuliaCon, and we wondered whether there aren’t cases where the compiler can at least turn x into a Ref{Int} instead of a Box.

Is there an existing discussion about efforts to do this kind of analysis in the compiler?

CC: @oxinabox, @Syx_Pek, @Oscar_Smith, @oschulz

For example, in this case we’d expected the compiler to deduce x as a Ref{Int}:

julia> function closure_surprise()
           x::Int = 0
           # I would've expected the closure to be "constructed" here, with the value in `x`
           # captured by copy at _this point_.
           f = ()->println("INNER: $x")

           x = 1
           return f
       end
closure_surprise (generic function with 1 method)

julia> f = closure_surprise()
#4 (generic function with 1 method)

julia> dump(typeof(f))
var"#4#5" <: Function
  x::Core.Box

julia> @code_warntype closure_surprise()
Variables
  #self#::Core.Const(closure_surprise, false)
  #4::var"#4#5"
  f::var"#4#5"
  x::Core.Box

Body::var"#4#5"
  ...
4 Likes
#10

The issue is that closure conversion is done in lowering which happens well before any type inference. So looking at @code_warntype gives a false sense of what the compiler knows when it’s generating the closure struct.

In the current compiler I imagine it may be possible to support this particular case and generate a Ref rather than a box because the typeassert is purely syntactic (and therefore available to the code in lowering).

But in general it’s hard to attack the boxing problem systematically without moving some part of closure conversion out of lowering and making it happen during type inference/optimization. For example, https://github.com/JuliaLang/julia/pull/31253

2 Likes