Closures magic and how stable?

General Usage
,
1

There is a feature in Julia that has enabled us to do some magical things. It is all thanks to this:

julia> adder(f, a) = ()-> f() + a
adder (generic function with 1 method)

julia> f = ()->5
#3 (generic function with 1 method)

julia> g = adder(f, 1)
#1 (generic function with 1 method)

julia> h = adder(f, 2)
#1 (generic function with 1 method)

julia> typeof(g) == typeof(h)
true

The fact that g and h are the same type has enabled us to do some truly magical things. Our code basically takes in some user input parameters as numbers, processes them a bit with arithmetic/math operations, and then passes those resulting numbers to a CPU/GPU kernel that simulates charged particles. The kernel will be JIT compiled of course based on the types of the resulting numbers. One feature I added was the ability to make the user input numbers actually be functions of time, which are then evaluated inside of the kernel itself per-particle. A special type wrapping an anonymous function was created, and arithmetic operations overloaded so it acts like a number - this is for the processing step. Then we pass the resulting function to the kernel, which is JIT compiled for said anonymous function. Thanks to this above feature, the kernel only needs to be JIT-compiled once, not every time it is called with the same user input (the processing step with overloaded arithmetic operators does not create new anonymous function types each time). It is really amazing, and we tested it and it works with CUDA etc.

As such, I just wanted to verify that this feature above is considered a stable part of the language?

2

Yes.

3 Likes
3

Yay

4

Without this, various important language constructs (e.g. map, mapreduce, open, etc. etc.), and gigantic swathes of the ecosytem would be practically unusable.

1 Like
5

It is a very clever solution, and I guess I had to ask because it almost seems too good to be true

6

I wonder why in the OP use case, using an explicit struct is not preferable to an anonymous function:

struct Adder{F, A}
    f::F
    a::A
end

(adder::Adder)() = adder.f() + adder.a

f() = 5
g = Adder(f, 1)
h = Adder(f, 2)
typeof(g) == typeof(h) == Adder{typeof(f), Int}

But now Adder is an explicit type than can be tracked in the codebase.

7

I think not necessarily because the OP says

So they aren’t providing these closures, the user does, which makes the sitiation analogous to e.g. mapreduce.

8

If you want an official quote, the Types of functions section of the Manual’s Type page says:

Closures also have their own type, which is usually printed with names that end in #<number> . Names and types for functions defined at different locations are distinct, but not guaranteed to be printed the same way across sessions.

()-> f() + a defines a closure at one location (in the source code), thus one (generated) name and type.

2 Likes