# Help understand boxing

**URL:** <https://discourse.julialang.org/t/help-understand-boxing/111829>\
**Category:** Performance\
**Created:** [March 19, 2024, 4:36pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829 "2024-03-19T16:36:18Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![jlamb](https://avatars.discourse-cdn.com/v4/letter/j/7ba0ec/32.png) [@jlamb](https://discourse.julialang.org/u/jlamb)\
**Post date:** [March 19, 2024, 4:36pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/1 "2024-03-19T16:36:18Z")

</div>

I am trying to resolve a type stability problem in a function where some local variables are showing up as the type `Core.Box` when I run `@code_warntype`. I’ve narrowed the cause to where I use a comprehension or generator with those variables as inputs, _and_ those variables can be modified later in the function. ’

Here’s a MRE. In the second function `NoBox()` with the line `x = 1.0` commented out, `x` is correctly reported as `Float64`.

```julia
function box()

    x::Float64 = rand()
    y = 0.0

    if x > 0
        y = mean((round(x) for _ in 1:100))
    end

    x = 1.0

    return(x, y)

end

function nobox()

    x::Float64 = rand()
    y = 0.0

    if x > 0
        y = mean((round(x) for _ in 1:100))
    end

    #x = 1.0

    return(x, y)

end

```

As a workaround, if I move the equivalent in my code of the line `y = mean((round(x) for _ in 1:100))` to its own function, it also works as expected and runs several times faster.

I usually try to break up my code in a way that is pretty natural to function barriers, but this originally seemed like too trivial a step to separate out. I’d appreciate any insight into why this is happening and if there’s a proper way to do this.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [March 19, 2024, 4:58pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/2 "2024-03-19T16:58:14Z")

</div>

This is so confusing for me too:

```julia
julia> function box()
           x::Float64 = rand()
           y = 0.0
           if x > 0
               y2 = mean((round(x) for _ in 1:100))
           end
           x = 1.0
           return(x, 1)
       
       end
box (generic function with 1 method)

julia> function no_box()
           x::Float64 = rand()
           y = 0.0
           #if x > 0
           # y2 = mean((round(x) for _ in 1:100))
           #end
           x = 1.0
           return(x, 1)
       
       end
no_box (generic function with 1 method)

```

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [March 19, 2024, 5:01pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/3 "2024-03-19T17:01:31Z")

</div>

I believe this is the famous variable captured in closure bug.

---

<div class="post-metadata">

**Author:** ![Zentrik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zentrik/32/35409_2.png) [@Zentrik](https://discourse.julialang.org/u/Zentrik)\
**Post date:** [March 19, 2024, 5:02pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/4 "2024-03-19T17:02:16Z")

</div>

[Performance Tips · The Julia Language](https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured) has some more information about this particular problem.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [March 19, 2024, 5:34pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/5 "2024-03-19T17:34:07Z")

</div>

Yeah but in this case the type annotation does not help.

I’m aware of the captured variable bug but it’s kinda annoying that it’s so re-occurring. Here it seems that the comprehension captures `x`?

What helps is.

```julia
 function box_fixed()
     x::Float64 = rand()
     y = 0.0
 
     let x=x
          if x > 0
              y2 = mean((round(x) for _ in 1:100))
          end
     end
     x = 1.0
     return(x, 1)
 end

```

I often encountered cases where this has huge performance penalties and as a user this is really worrisome.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [March 19, 2024, 5:40pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/6 "2024-03-19T17:40:02Z")

</div>

Correct, `Meta.@lower` is your friend here:

```julia-repl
julia> Meta.@lower mean(x for _ in 1:100)
:($(Expr(:thunk, CodeInfo(
    @ none within `top-level scope`
1 ─ $(Expr(:thunk, CodeInfo(
    @ none within `top-level scope`
1 ─ global var"#5#6"
│ const var"#5#6"
│ %3 = Core._structtype(Main, Symbol("#5#6"), Core.svec(), Core.svec(), Core.svec(), false, 0)
│ Core._setsuper!(%3, Core.Function)
│ var"#5#6" = %3
│ Core._typebody!(%3, Core.svec())
└── return nothing
)))
│ %2 = Core.svec(var"#5#6", Core.Any)
│ %3 = Core.svec()
│ %4 = Core.svec(%2, %3, $(QuoteNode(:(#= none:0 =#))))
│ $(Expr(:method, false, :(%4), CodeInfo(
1 ─ return x
)))
│ #5 = %new(var"#5#6")
│ %7 = #5
│ %8 = 1:100
│ %9 = Base.Generator(%7, %8)
│ %10 = mean(%9)
└── return %10
))))

```

IMO, this is one of the most surprising and “leakiest” parts of lowering. There is no indication that a closure is created when you write a comprehension, so people don’t know to look for one when trying to hunt down the source of the box. Ideally, there would be a way to stabilize captures in comprehensions which doesn’t require solving the entirety of the closure capture problem.

---

<div class="post-metadata">

**Author:** ![jlamb](https://avatars.discourse-cdn.com/v4/letter/j/7ba0ec/32.png) [@jlamb](https://discourse.julialang.org/u/jlamb)\
**Post date:** [March 19, 2024, 5:59pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/7 "2024-03-19T17:59:37Z")

</div>

> There is no indication that a closure is created when you write a comprehension, so people don’t know to look for one when trying to hunt down the source of the box.

This is certainly part of my confusion. I was trying to comprehend (no pun intended) how the documentation linked by @Zentrik related to this since I did not think I was creating or returning a sub-function but the evaluated result.

---

<div class="post-metadata">

**Author:** ![jlamb](https://avatars.discourse-cdn.com/v4/letter/j/7ba0ec/32.png) [@jlamb](https://discourse.julialang.org/u/jlamb)\
**Post date:** [March 19, 2024, 6:05pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/8 "2024-03-19T18:05:21Z")

</div>

Well, that certainly feels awkward and counterintuitive. I’ll stick to separating out the function. It’s a shame because comprehensions, generators, map, etc. are all such workhorses.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [March 19, 2024, 6:56pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/9 "2024-03-19T18:56:41Z")

</div>

I do wonder whether we could fix this by lowering to opaque closures. this is a case where we make the closure and immediately execute it, so making a closure does lose information.

---

<div class="post-metadata">

**Author:** ![sgaure](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sgaure/32/14779_2.png) [@sgaure](https://discourse.julialang.org/u/sgaure)\
**Post date:** [March 19, 2024, 9:59pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/10 "2024-03-19T21:59:28Z")

</div>

> [@Oscar\_Smith](#):
>
> I do wonder whether we could fix this by lowering to opaque closures. this is a case where we make the closure and immediately execute it, …

Unless, of course, you keep the generator around:

```julia
function box()
    x::Float64 = rand()
    if x > 0
        y = (round(x) for _ in 1:10)
    end
    x = 1.0
    return y
end

collect(box())

```

return a vector of `1.0`s.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [March 19, 2024, 11:37pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/11 "2024-03-19T23:37:41Z")

</div>

> [@ToucheSir](#):
>
> There is no indication that a closure is created when you write a comprehension, so people don’t know to look for one when trying to hunt down the source of the box.

> [@roflmaostc](#):
>
> Here it seems that the comprehension captures `x`?

The [scoping rules](https://docs.julialang.org/en/v1/manual/variables-and-scoping/#scope-of-variables) say that comprehensions make new local scopes, which use outer local variables; they are built on methods so that requires capturing. `Expr(:method...` showed up in the lowered code to make the closure. Closure behavior and the performance pitfalls are plainly documented in the sections on comprehensions and generator expressions in [Single- and multi-dimensional Arrays · The Julia Language](https://docs.julialang.org/en/v1/manual/arrays/#man-comprehensions).

> [@roflmaostc](#):
>
> ```julia
> let x=x
> if x > 0
> y2 = mean((round(x) for _ in 1:100))
> end
> end
> 
> ```

If the value of `x` at a particular point is preferable to capturing the variable `x` in general, then this is semantically the proper way.

> [@Oscar\_Smith](#):
>
> variable captured in closure bug.

I think this is a fair description because there are feasible ways to mitigate this, but “bug” doesn’t get across the difficulty of mixing apparently straightforward type inference with performant closure implementation.

For example, sgaure just pointed out that immediate execution for a particular value cannot be an automatic optimization if the generator persists. Unfortunately, even `y = mean((round(x) for _ in 1:100))` does not guarantee that the generator is never reused because `mean` could cache the generator somewhere for all we know. I assume some compiler effect analysis can address that, but I’m not sure how reliable that is, and it’s moot because boxing is still determined at lowering, far before a call signature is available to trigger compilation. We’d prefer something that works unconditionally and safely.

---

<div class="post-metadata">

**Author:** ![alfaromartino](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alfaromartino/32/52986_2.png) [@alfaromartino](https://discourse.julialang.org/u/alfaromartino)\
**Post date:** [March 19, 2024, 11:51pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/12 "2024-03-19T23:51:01Z")

</div>

My recommendation in this respect is not to reassign local variables or function arguments inside a function. It can introduce unexpected behavior in several instances, given the issue of variables captured in a closure.

Example:

```julia
function foo()
    x = [1.0, 2.0]
    x = [1, 2]

    [mean(x) for _ in 1:10]
end

@code_warntype foo() #type unstable

```

Thus, the following is fine:

```julia
function nobox()
    x = rand()
    y = 0.0

    if x > 0
        y = mean(round(x) for _ in 1:100)
    end

    z = 1.0 # don't reassign`x`

    return z, y

end

```

---

<div class="post-metadata">

**Author:** ![jlamb](https://avatars.discourse-cdn.com/v4/letter/j/7ba0ec/32.png) [@jlamb](https://discourse.julialang.org/u/jlamb)\
**Post date:** [March 20, 2024, 3:11pm UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/13 "2024-03-20T15:11:45Z")

</div>

> My recommendation in this respect is not to reassign local variables or function arguments inside a function.

I appreciate that. The example I provided does not convey what I’m doing with the real code, where the final value of `x` is constrained by an intermediate evaluation of `y(x)`. A little more specifically I am solving and returning a point `<x, y(x), ... >` where `x` and other dimensions must be adjusted to zero for certain ranges of `y(x)`.

There may indeed be cleaner ways of writing this, and it does work fine to tuck `y(x)` into a separate function. What’s confusing being new to Julia is that `y(x)` introduces type stability for `x` at all since it isn’t modified at all within the comprehension.

---

<div class="post-metadata">

**Author:** ![alfaromartino](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alfaromartino/32/52986_2.png) [@alfaromartino](https://discourse.julialang.org/u/alfaromartino)\
**Post date:** [March 21, 2024, 1:30am UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/14 "2024-03-21T01:30:36Z")

</div>

Note that the comprehension is not introducing the type instability. It’s the fact of redefining a variable AND using a closure.

For instance, the second function is type unstable once you add a function definition inside a function, even if you type annotate `x`.

```julia
function foo()
    x = 1
    x = 1
        
    return x
end

@code_warntype foo() # type stable

function foo()
    x::Int64 = 1
    x = 1
    bar()::Int64 = x::Int64
    
    return bar()
end

@code_warntype foo() # type unstable

```

I know it’s quite confusing and the issue has been around for a long time. This hints that there’s no easy solution to the problem.

You can address the problem by either i) splitting a function into multiple separate functions, ii) not reassigning variables, iii) including all variables as arguments inside the closure (even functions).

```julia
# example of i)
bar(x) = x

function foo()
    x = 1
    x = 1    
    
    return bar(x)
end

@code_warntype foo() # type stable

# example of ii)
function foo()
    x = 1
    z = 1
    bar() = z
    
    return bar()
end

@code_warntype foo() # type stable

# example of iii)
function foo()
    x = 1
    x = 1
    bar(x) = x
    
    return bar(x)
end

@code_warntype foo() # type stable

```

---

<div class="post-metadata">

**Author:** ![alfaromartino](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alfaromartino/32/52986_2.png) [@alfaromartino](https://discourse.julialang.org/u/alfaromartino)\
**Post date:** [March 21, 2024, 1:35am UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/15 "2024-03-21T01:35:12Z")

</div>

This [link](https://alfaromartino.github.io/julia_book/PAGES/7c_gotchas/) could help you.

It’s from a book I’m writing with the basics of Julia. Note that the book is unfinished in terms of content, writing, etc (the link is not even public yet). Hopefully, a preliminary version will be ready by mid/end of the year.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 22, 2024, 11:24am UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/16 "2024-03-22T11:24:28Z")

</div>

4 posts were split to a new topic: [Formatting code examples in text with Franklin.jl](https://discourse.julialang.org/t/formatting-code-examples-in-text-with-franklin-jl/111960)

---

<div class="post-metadata">

**Author:** ![joa-quim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joa-quim/32/227_2.png) [@joa-quim](https://discourse.julialang.org/u/joa-quim)\
**Post date:** [March 22, 2024, 12:09am UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/20 "2024-03-22T00:09:32Z")

</div>

Thanks, I also found a way (see result [here](https://www.generic-mapping-tools.org/GMTjl_doc/geophysics/earthtides/01_earthtides/#the_3_components)) but is also very cumbersome.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [March 22, 2024, 10:20am UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/21 "2024-03-22T10:20:36Z")

</div>

Please note that `(round(x) for _ in 1:100)` is a generator with a closure, and `[round(x) for _ in 1:100]` is a comprehension which immediately executes its [implicit] generator.

> [@roflmaostc](#):
>
> in this case the type annotation does not help.

Actually, the type-annotation does help. If you look at the lowered code, you will notice that `x` is `typeassert`ed to `Float64` every time it is accessed. This means that the code is type-stable, and dynamic dispatch does not occur when calling functions on `x`.

This can be confirmed by looking at `@code_warntype box()`: although accessing the box contents retrieves a value of type `Any`, it is immediately `typeassert`ed to `Float64` and all subsequent calls are type-stable.

As a result, the code is far more performant than if the type-annotation wasn’t there: `box` has the same performance as `nobox`, other than the allocation required for the `Core.Box`. Without the type annotation, it would be about 100x slower.

> [@Oscar\_Smith](#):
>
> variable captured in closure bug.

In this case, this is not a bug; the creation of a box is semantically necessary since it’s impossible for lowering to know syntactically whether `mean` will store the generator or not.

We could make the `Core.Box` type-parameterized (similar to `Ref`), which allows additional compiler optimizations to work, but this has some challenges. I want to [solve those challenges](https://github.com/JuliaLang/julia/pull/53385#issuecomment-1965624891) but life’s been in the way.

> [@alfaromartino](#):
>
> Example:

In your example, the insertion of a box _is_ a “bug” (in the sense that `x` doesn’t actually need to be boxed to meet language semantics, but it’s boxed anyway because of how the capture logic is currently implemented).

It’s my opinion that it’s feasible to fix this bug, and I opened [an issue for it](https://github.com/JuliaLang/julia/issues/53367), but it might be an intensive affair to edit lowering without breaking other things.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [March 22, 2024, 10:21am UTC](https://discourse.julialang.org/t/help-understand-boxing/111829/22 "2024-03-22T10:21:37Z")

</div>

> [@ToucheSir](#):
>
> Ideally, there would be a way to stabilize captures in comprehensions which doesn’t require solving the entirety of the closure capture problem.

> [@Benny](#):
>
> If the value of `x` at a particular point is preferable to capturing the variable `x` in general, then this is semantically the proper way.

There’s an optimization for comprehensions that’s possible but currently is not implemented, since comprehensions use special syntax and are known to immediately execute their generator.

When we intend to capture a variable’s instantaneous value, the `let` block is indeed the proper way to go. `FastClosures` implements a macro which automates the creation of `let` blocks. Perhaps this could be edited to work also on generator expressions. If the macro was given a better name (currently it’s called `@closure`, which does not accurately reflect what it does), perhaps it could be part of Base.

If that is done, then lowering could emit a call to that macro whenever it encounters a comprehension.