# Define data-dependent function, various ways

**URL:** <https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988>\
**Category:** New to Julia\
**Tags:** closure, functors\
**Created:** [October 25, 2020, 6:15pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988 "2020-10-25T18:15:58Z")\
**Posts on this page:** 15\
**Page:** 2

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [October 27, 2020, 12:44pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/21 "2020-10-27T12:44:37Z")

</div>

```julia
julia> f(x) = x^2
f (generic function with 1 method)

julia> f = y -> y^2
ERROR: invalid redefinition of constant f
Stacktrace:
 [1] top-level scope at REPL[2]:1

julia> g = y -> y^2
#3 (generic function with 1 method)

julia> g = z -> z^2
#5 (generic function with 1 method)

```

One is a named generic function. The other is an anonymous function bound to a variable. Nothing involved is `const`, except insofar as one could act like the named generic function is `const` since it can’t be easily rebound.

The `const` concern is distinct from the named generic function vs. anonymous function distinction.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [October 27, 2020, 2:01pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/22 "2020-10-27T14:01:02Z")

</div>

I want to chime in here in support of @lmiq. Understanding closure performance is not easy, even for experienced users. There have been various threads on Discourse and Github that discuss performance problems with closures, which is why [FastClosures.jl](https://github.com/c42f/FastClosures.jl) was created. For example, there is this open issue on Github:

> <https://github.com/JuliaLang/julia/issues/15276>
>
> \`\`\` jl
> using Images: realtype
> 
> function ifi{T\<:Real,K,N}(img::AbstractArray{T,N}…, kern::AbstractArray{K,N}, border::AbstractString, value)
> if border == "circular" && size(img) == size(kern)
> out = real(ifftshift(ifft(fft(img).\*fft(kern))))
> elseif border != "inner"
> prepad = \[div(size(kern,i)-1, 2) for i = 1:N\]
> postpad = \[div(size(kern,i), 2) for i = 1:N\]
> fullpad = \[nextprod(\[2,3\], size(img,i) + prepad\[i\] + postpad\[i\]) - size(img, i) - prepad\[i\] for i = 1:N\]
> A = padarray(img, prepad, fullpad, border, convert(T, value))
> krn = zeros(typeof(one(T)\*one(K)), size(A))
> indexesK = ntuple(d-\>\[size(krn,d)-prepad\[d\]+1:size(krn,d);1:size(kern,d)-prepad\[d\]\], N)::NTuple{N,Vector{Int}}
> AF = ifft(fft(A).\*fft(krn))
> out = Array(realtype(eltype(AF)), size(img))
> end
> out
> end
> \`\`\`
> 
> Test:
> 
> \`\`\` jl
> julia\> @code\_warntype ifi(rand(3,3), rand(3,3), "replicate", 0)
> Variables:
> #self#::#ifi
> img::Array{Float64,2}
> kern::Array{Float64,2}
> border::ASCIIString
> value::Int64
> prepad::Box
> ...
> \`\`\`
> 
> Now comment out the \`indexesK = ...\` line (the output of which is not used at all). Suddenly \`prepad\` is inferred as \`Array{Int, 1}\`.

I haven’t sat down and studied the scenarios where closures are performant and the scenarios where closures are not performant, so I often wonder if I should be reaching for FastClosures.jl. Is it really just as simple as the following?

> [@stevengj](#):
>
> Avoiding non- `const` global variables in inner loops is the one and only rule here.

Recently there was a [question about closures on Stack Overflow](https://stackoverflow.com/questions/64512168/julia-closure-with-a-let-block-is-this-kosher). In Bogumil’s answer, he pointed out that two different closures were type unstable. For example, this is type unstable:

```julia
counter = let
    local counter = 0 # local variable
    f() = counter += 1 # returned function
end

```

But I have _no idea_ why that is type unstable.

If nothing else, we have a documentation problem. It would be nice if there was a section of the manual that very clearly spelled out which situations create slow or type unstable closures and which do not.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [October 27, 2020, 2:11pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/23 "2020-10-27T14:11:55Z")

</div>

> [@CameronBieganek](#):
>
> It would be nice if there was a section of the manual that very clearly spelled out which situations create slow or type unstable closures and which do not.

I think we actually already have that. The case you’ve brought up here is covered in the performance tips here: [Performance Tips · The Julia Language](https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured) and the issue with global variables is in the performance tips here: [Performance Tips · The Julia Language](https://docs.julialang.org/en/v1/manual/performance-tips/#Avoid-global-variables)

The reason that the global variables tip doesn’t mention closures in particular is that it applies to _all_ Julia code, not just closures. Any access of a non-const global variable will be slower and harder to type-infer than code which does not use a non-const global variable.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 27, 2020, 2:21pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/24 "2020-10-27T14:21:45Z")

</div>

> [@johnmyleswhite](#):
>
> having a page with guides on the _best way_ to write Julia code for this use case?

The manual has a bit on [callables](https://docs.julialang.org/en/v1/manual/methods/#Function-like-objects), with an example.

---

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [October 27, 2020, 2:31pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/25 "2020-10-27T14:31:02Z")

</div>

The manual is not what people are missing – they’re looking for a cookbook that maps specific problems they’re working on to end-to-end solutions.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 27, 2020, 2:39pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/26 "2020-10-27T14:39:05Z")

</div>

> [@johnmyleswhite](#):
>
> they’re looking for a cookbook that maps specific problems they’re working on to end-to-end solutions

Yes, I am aware of this. But it is an impossible problem: the space of specific problems is simply too large to even navigate, let alone provide a solution for.

It is much better to invest into learning about building blocks. Examples should help with this, but are not meant to be an exhaustive dictionary of all possible problems.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 27, 2020, 3:25pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/27 "2020-10-27T15:25:41Z")

</div>

> [@rdeits](#):
>
> it applies to _all_ Julia code, not just closures

The fact that the variable to which a function is bound is _just a variable_ is not obvious in general. These things are not common in some languages (Fortran, for example). This is the main origin of my confusions.

> [@CameronBieganek](#):
>
> It would be nice if there was a section of the manual that very clearly spelled out which situations create slow or type unstable closures and which do not.

Concerning additional documentation: I more or less go with the ones that think this is impossible. Essentially I think that these threads _are that_. For instance, in Python, if one searches for a solution to a problem, it is almost certain that we reach a stack overflow or other discussion, and not any official Python documentation. The documentation of Julia is pretty fantastic in my opinion and, while it can be improved in some specific points, the fact is that it will not be the most common place in which a search will end as the language becomes more popular and specific problems and use cases become covered by questions and answers here.

That said, I suggested in some other thread that if we had an infrastructure _within_ discourse to produce didatic material and develop programming curses, that material and the questions and answers that each teaching effort produces would end up collaborating for the production of this example-based docs. For example, I am giving a programming course now (in Julia, for chemists that mostly do not have much programming experience), and I am producing content, like this (unfortunately in Portuguese):

[https://github.com/m3g/CKP/blob/master/disciplina/codes/Int-vs-Any.md](https://github.com/m3g/CKP/blob/master/disciplina/codes/Int-vs-Any.md)

and this: [https://github.com/m3g/CKP/blob/master/disciplina/simulacoes2.pdf](https://github.com/m3g/CKP/blob/master/disciplina/simulacoes2.pdf)

Much of the topics I end up writing with detail come from the fact that some students report some issues, some of them which I can answer directly, some of them that actually lead me to understand better the language, many times after coming here to ask for help. Yet, this material will probably not be very visible to someone outside the course. If it was part of discourse in some way, with possible eventual inputs of other people, or stimulating discussions among the students of the course themselves, that could have another impact.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [October 27, 2020, 3:46pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/28 "2020-10-27T15:46:16Z")

</div>

> [@lmiq](#):
>
> The fact that the variable to which a function is bound is _just a variable_ is not obvious in general. These things are not common in some languages (Fortran, for example). This is the main origin of my confusions.

Oh yeah, I don’t mean to suggest that it should be obvious. It’s just hard to know exactly where to make that particular point so that users who need to learn it will be receptive to it while users who do not will not be distracted by it. For example, precisely the same notion applies to types which likewise cannot be bound to variables in many languages.

There is an unusual feature of Julia here, where `struct Foo` or `function foo` are slightly more special than `Foo = ...` or `foo = ...` in that the former create constants while the latter do not. Maybe that’s the fundamental point we need to make more clear?

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 27, 2020, 4:03pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/29 "2020-10-27T16:03:30Z")

</div>

> [@rdeits](#):
>
> There is an unusual feature of Julia here, where `struct Foo` or `function foo` are slightly more special than `Foo = ...` or `foo = ...` in that the former create constants while the latter do not. Maybe that’s the fundamental point we need to make more clear?

Clearly that is something that was not completely clear to me. Now, I guess that is written in the manual somewhere. I am not sure if we can guarantee that, being written anywhere else, it will get read specifically by someone that is facing the same confusion. The confusions have the property of taking us to the wrong places of a user manual as well. We quickly find what we are searching for in a manual when we almost know exactly what we want.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [October 27, 2020, 4:11pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/30 "2020-10-27T16:11:32Z")

</div>

> [@rdeits](#):
>
> The case you’ve brought up here is covered in the performance tips here: [Performance Tips · The Julia Language](https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured)

Thanks for the link to the “Performance of captured variables” section. I read that a while ago, but it looks like that section has been expanded a lot since I read it, so I will take another look.

Part of the point I’m making is that performance of closures _can_ be subtle, and comments from experts like “it’s not tricky at all” are off-putting and not very helpful.

---

<div class="post-metadata">

**Author:** ![tomohiro\_soejima](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomohiro_soejima/32/8056_2.png) [@tomohiro\_soejima](https://discourse.julialang.org/u/tomohiro_soejima)\
**Post date:** [October 31, 2020, 9:45am UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/31 "2020-10-31T09:45:47Z")

</div>

Just wanted to quickly add that the main issue here is the fact that each invocation of `x->f_closure(x, data)` creates a **new** function.

```julia
f = x->f_closure(x,data)
g = x->f_closure(x,data)
@show f == g # false

```

When you call `s_outer(x->f_closure(x,data), x0)` multiple times, it needs to compile each time because the inputs are different functions. You can, however, circumvent this problem by assigning your anonymous function to a variable.

```julia
func_test = x -> f_closure(x, data)
@time s_outer(func_test,x0)

    0.002346 seconds (984 allocations: 65.018 KiB)

@time s_outer(func_test,x0)

    0.000002 seconds (1 allocation: 16 bytes)

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 31, 2020, 10:22am UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/32 "2020-10-31T10:22:45Z")

</div>

> [@tomohiro\_soejima](#):
>
> it needs to compile each time because the inputs are different functions

It is best to create these kind of closures within functions, not at the REPL, then AFAIK they won’t need to be compiled again (for the same types).

---

<div class="post-metadata">

**Author:** ![tomohiro\_soejima](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomohiro_soejima/32/8056_2.png) [@tomohiro\_soejima](https://discourse.julialang.org/u/tomohiro_soejima)\
**Post date:** [October 31, 2020, 10:32am UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/33 "2020-10-31T10:32:17Z")

</div>

Agreed that these closures should be defined within functions to avoid precisely these issues. I just point out out the initial comparison was probably not doing what he intended to do because of the issue I raised. (I for one was confused why closure could not get compiled while `f_global_const_data` could be compiled just fine).

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [October 31, 2020, 11:42am UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/34 "2020-10-31T11:42:57Z")

</div>

While searching for an introduction to closures, I came across this clear explanation in [Julia Language Pedia (Introduction to Closures).](https://julia-lang.programmingpedia.net/en/tutorial/5724/closures) No clue on how that “Juliapedia” was put together, the content is excellent but unfortunately it seems to have stopped being updated around 2017? They have a disclaimer saying the material is an extract of the original Stack Overflow Documentation… If there was a way to further build upon all of Julia’s knowledge and solutions already available in discourse, that would be of great service.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 2, 2020, 1:14pm UTC](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/35 "2020-11-02T13:14:50Z")

</div>

> [@rafael.guerra](#):
>
> If there was a way to further build upon all of Julia’s knowledge and solutions already available in discourse, that would be of great service.

I think that closures in particular should have a short explanation in the _manual_, which uses the term in various places.

Julia’s closures are not conceptually different from closures in other languages that have them, so those who have encountered the concept do not need a Julia-specific introduction, but for the rest of the readers a paragraph with an example and then an explanation how they related to [function-like objects](https://docs.julialang.org/en/v1.6-dev/manual/methods/#Function-like-objects) could be helpful.

[Previous page](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988.md?page=1)