# Should closures be avoided?

**URL:** <https://discourse.julialang.org/t/should-closures-be-avoided/96073>\
**Category:** General Usage\
**Tags:** question, closure\
**Created:** [March 14, 2023, 5:51pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073 "2023-03-14T17:51:29Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![prittjam](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/prittjam/32/21267_2.png) [@prittjam](https://discourse.julialang.org/u/prittjam)\
**Post date:** [March 14, 2023, 5:51pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/1 "2023-03-14T17:51:30Z")

</div>

There seem to be several performance-related issues with closures (boxed variables) that can be tricky to understand when they will arise. Should closures be avoided as a rule? What is an alternative design pattern? They are very convenient and readable in general.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [March 14, 2023, 5:57pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/2 "2023-03-14T17:57:34Z")

</div>

Please read the [performance tips](https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured) which discusses this. You can recover full performance by capturing variables manually in a `let` block.

---

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [March 14, 2023, 7:28pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/3 "2023-03-14T19:28:49Z")

</div>

If the code is not a performance bottleneck, especially in one-off scripts, I’ll use closures and indulge in type instabilities. That’s the benefit of using a dynamically typed language. Not all code needs to be hyper-optimized.

---

<div class="post-metadata">

**Author:** ![Norman](https://avatars.discourse-cdn.com/v4/letter/n/97f17d/32.png) [@Norman](https://discourse.julialang.org/u/Norman)\
**Post date:** [March 14, 2023, 7:49pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/4 "2023-03-14T19:49:52Z")

</div>

Just to add to the answers, another option is to define a callable struct as shown [here](https://docs.julialang.org/en/v1/manual/methods/#Function-like-objects) and create an instance of the struct instead of the anonymous function. This is essentially making an explicit definition on what the closure is.

---

<div class="post-metadata">

**Author:** ![Salmon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/salmon/32/22968_2.png) [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Post date:** [March 14, 2023, 8:08pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/5 "2023-03-14T20:08:27Z")

</div>

Isn’t it sufficient to check with @code\_warntype, whether any variables are boxed and eliminate if neccessary?  
Or are there other reasons why closures might be slower?  
The way I see it, the pitfall is not as bad as it sounds.  
Write your code in the obvious way. Then check if it is performant enough. If not, eliminate all type instabilities.  
Unless I am missing something, the correct usage of closures doesn’t require you to always be aware of where exactly difficulties can arize.

---

<div class="post-metadata">

**Author:** ![prittjam](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/prittjam/32/21267_2.png) [@prittjam](https://discourse.julialang.org/u/prittjam)\
**Post date:** [March 14, 2023, 8:16pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/6 "2023-03-14T20:16:07Z")

</div>

I write a lot of optimization code that requires different fitting, loss, and weighting functions that plug into a robust optimizer. These functions have to be fast, but they also need to be convenient since we are often testing new ideas. So my use case is a “worst-case” scenario for the closure boxing issue. I read elsewhere that other people writing optimization code were also encountering this issue. I was writing callable structs as @Norman suggested, but it creates a lot of boilerplate code. I suppose judicious use of FastClosures might help, but I don’t understand the implications of that macro. @code\_warntype checking is also not a panacea although it is suggested that new versions of Cthulu might be easier to use.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [March 14, 2023, 8:48pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/7 "2023-03-14T20:48:44Z")

</div>

Indeed, closures cause bad performance far more often than I’d like—especially because of how nice the syntax for comprehensions and generators is, and how common the pattern of assigning variables inside conditionals is. IMO the objective should be to [improve the boxing situation](https://discourse.julialang.org/t/rfc-some-ideas-to-tackle-15276-performance-of-captured-variables-in-closures/95260), but I can’t blame anyone for following a heuristic of avoiding closures until that happens.

There was a similar thread a few days ago; do you find this helpful?

> [@Are "Closures should be avoided whenever possible" still valid in Julia v1.9+?](https://discourse.julialang.org/t/are-closures-should-be-avoided-whenever-possible-still-valid-in-julia-v1-9/95893/5):
>
> There are two scenarios where Fix1 and Fix2 (and possibly [generic typed partial applicators](https://github.com/uniment/PartialFuns.jl), if humanity survives long enough to see them) offer benefit: To avoid recompilation. To avoid boxing when variables are captured. To illustrate the first benefit—avoiding recompilation—consider the following example: # After warmup julia\> @time let xs=(1, 2.0, 3+0im) reduce((x,y)-\>x+y, map(x-\>x^2, Iterators.filter(x-\>isodd(x), xs))) end; 0.304171 seconds (124.78 k allocations: 8.…

---

<div class="post-metadata">

**Author:** ![prittjam](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/prittjam/32/21267_2.png) [@prittjam](https://discourse.julialang.org/u/prittjam)\
**Post date:** [March 15, 2023, 1:55pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/8 "2023-03-15T13:55:20Z")

</div>

I would say at this point I’m focused on mathematical correctness but I have noticed the type instabilities proliferating with the use of closures and when I put functions into structure members (even when parameterized). I don’t really understand why, and I don’t really have a big time budget to dig into the more subtle issues, unfortunately. I guess there are two packages meant to deal with this: FastClosures and FunctionWrappers. I will need to fix it eventually to achieve state-of-the-art runtime. I’m worried that despite using fairly idiomatic code that I will have to tear it apart to get the speed needed.

I am competing against C++ frameworks which are hand-tuned. While we will have an algorithmic win, it won’t matter if I can’t close the performance gap in wall clock time. (We are trying to use Julia to publish). I’ve done static polymorphism (compiile-time) in C++ (expression templates) Eigen, etc… and there you know when you write your code it will be fast. I would say that if you stick to the design patterns; it is also elegant. The drawback with C++ is you lose a reasonable REPL. But I wonder what the time tradeoff will be between chasing down type instabilities and slower development time in C++.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [March 15, 2023, 2:36pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/9 "2023-03-15T14:36:28Z")

</div>

> [@prittjam](#):
>
> I’m worried that despite using fairly idiomatic code that I will have to tear it apart to get the speed needed.

Having functions encoded as part of the type is not really idiomatic code. Maybe a MWE of a what you are doing and your performance pitfalls would be helpful?

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [March 15, 2023, 3:42pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/10 "2023-03-15T15:42:46Z")

</div>

Once you know what to avoid, closures are usually fast. I use them in high performance inner loops pretty much everywhere. But there are few other gotchas to closures, like If you pass a variable `Type` into a closure you lose type stability as it is stored as a field in the anonymous struct.

If you share some code for your unstable closures you may get better specific feedback.

---

<div class="post-metadata">

**Author:** ![prittjam](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/prittjam/32/21267_2.png) [@prittjam](https://discourse.julialang.org/u/prittjam)\
**Post date:** [March 15, 2023, 3:47pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/11 "2023-03-15T15:47:54Z")

</div>

Yes, I think I will release the framework I’m working on to get some feedback.

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [March 15, 2023, 3:49pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/12 "2023-03-15T15:49:32Z")

</div>

You will get more feedback on small MWEs posted here than a framework

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [March 15, 2023, 7:31pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/13 "2023-03-15T19:31:13Z")

</div>

> [@prittjam](#):
>
> type instabilities proliferating with the use of closures and when I put functions into structure members (even when parameterized) … FunctionWrappers

If you use the `let` block [trick](https://github.com/JuliaLang/julia/issues/15276#issuecomment-297596373), then accessing the closure’s capture will be type-stable at least.

Out of curiosity, what’s your purpose for using FunctionWrappers?

---

<div class="post-metadata">

**Author:** ![prittjam](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/prittjam/32/21267_2.png) [@prittjam](https://discourse.julialang.org/u/prittjam)\
**Post date:** [March 17, 2023, 2:13pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/14 "2023-03-17T14:13:28Z")

</div>

```julia
typstruct Irls{D <: AbstractVector, F, R, S, W}
    data::D
    residuals::Vector{Float64}
    weights::Vector{Float64} 
    fit::F
    residual::R
    scale::S
    weight::W
end

Irls(data, fit, residual, scale, weight) = 
    Irls(data, similar(data,Float64), similar(data,Float64),
         fit, residual, scale, weight)

function (irls::Irls)(; kwargs...)
    num_data = length(irls.data)
    resize!.((irls.residuals, irls.weights), num_data)
    irls_fit() = irls.fit(irls.data)
    irls_fit(weights) = irls.fit(irls.data, weights)
    residuals = (location)->irls.residuals .= irls.residual.(irls.data, Ref(location))
    weights = (residuals, scale)->irls.weights .= irls.weight.(residuals,scale)
    
    return irls(irls_fit, residuals, weights, irls.scale; kwargs...)
end

```

Above is a structure that defines an iteratively reweighted least squares problem. the fitting function, residuals, weights, and scale estimation are all functions that define the irls behavior. The struct is holding the functions to be called in the irls loop. I define the problem by initializing the structure and then pass it on to the code that needs to use it.

I suspect that storing functions in a struct like this is causing dynamic dispatch or type instability in my code. I was going to try FuncitonWrappers to resolve it. at this point I’m not sure what is causing the type instability.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [March 17, 2023, 9:21pm UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/15 "2023-03-17T21:21:26Z")

</div>

> [@prittjam](#):
>
> I suspect that storing functions in a struct like this is causing dynamic dispatch or type instability in my code.

Hm, I don’t think so—the struct is being type-parameterized so this code appears to be type-stable afaict.

That said, if you’re storing such structs in an array, and if different structs are holding different fitting functions (by which I mean functions with different function bodies, not just closures with different captured values) then accessing the elements of that array will be type-unstable because it won’t have a concrete element type. To illustrate:

```julia
julia> z=[Irls([1,], x->x, [1,], 1, [1,]), Irls([1,], x->x, [1,], 1, [1,])]
2-element Vector{Irls{Vector{Int64}, F, Vector{Int64}, Int64, Vector{Int64}} where F}:
 Irls{Vector{Int64}, var"#56#58", Vector{Int64}, Int64, Vector{Int64}}([1], [6.13256783589e-312], [7.15252505199e-312], var"#56#58"(), [1], 1, [1])
 Irls{Vector{Int64}, var"#57#59", Vector{Int64}, Int64, Vector{Int64}}([1], [7.165628674093e-312], [6.11134787798e-312], var"#57#59"(), [1], 1, [1])

julia> eltype(z)
Irls{Vector{Int64}, F, Vector{Int64}, Int64, Vector{Int64}} where F

julia> isconcretetype(eltype(z))
false

```

(notice that the function types are different—`var"#56#58"` vs `var"#57#59"`; although the functions do the same thing, they’ve been declared separately.)

If indeed it’s an array access that’s type-unstable, then I think FunctionWrappers.jl should help, though I haven’t used it. Other options include manually setting the array’s element type as a narrow `Union` of types, or passing the array element through a [function barrier](https://docs.julialang.org/en/v1/manual/performance-tips/#kernel-functions) to infer its type before doing repetitive operations on it.

Note that this is just speculation until you can provide a MWE showing the type instability.

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [March 18, 2023, 10:52am UTC](https://discourse.julialang.org/t/should-closures-be-avoided/96073/16 "2023-03-18T10:52:32Z")

</div>

This is also a pretty extreme use of closures passed to closures calling functions from struct fields. You might see the compiler giving up and not tracking types through all of that.

`Cthulhu.@descend` may help understand whats happening.