What are Opaque Closures?

General Usage
1

In Slack there was a question about opaque closures. I found that what I was assuming they were is not the real thing, they are not closures that pass variables as value… so here I post the answer (thanks to @oxinabox) and a link to the original comment so it can be more visible.

As I understand it.
This is not true [1]. They do not do that.

Opaque closures are closures that make no promises about what is inside them.

Normal closures promise to carry references to every variable that is used inside them (and nothing else), and to run exactly the code inside them when they are called. Including respecting newly created or overwritten methods.

Opaque closures on the other hand relax the requirements, and basically just promise to perform something equivalent in behaviour to their body. Except they always run in the world age they were created, so they don’t observe new methods that were added (this makes them very unfriendly for interactive use, but like @pure they are not really intended for normal use).
What this allows them to do is move computation that depends only on variables in the parent scope (and not on the closures inputs) out of the closure and into the parent scope so they run when the closure is created (and then that result is closed over). This in turn makes them much more optimizable as the optimizer can optimize the (parts of) the closure body and the parent function together.

This has nothing to do with capture by name or capture by value.

  1. That they pass variables as value. ↩︎

9 Likes
2

I would like to better understand this point, so I tried to make an example.

function make_closure(y)
    foo(x) = x + 1

    function closure(x)
        foo(x) * y
    end

    return (foo, closure)
end

(foo, f) = make_closure(2)

julia> f(2)
6

julia> f(2.0)
6.0

julia> foo(x::Float64) = x + 1.25
ERROR: cannot define function foo; it already has a value
Stacktrace:
 [1] top-level scope
   @ none:0

julia> f.foo(x::Float64) = x + 1.25

julia> f(2)
6

julia> f(2.0)
6.5

So the closure f was sensitive to the new method added to foo. I’m not sure why it is not the case, but I did expect it to be possible to define a new method on the foo returned by make_closure.

It sounds like opaque closures would not allow

julia> f.foo
foo (generic function with 2 methods)

julia> f.y
2

which, from memory, is undocumented behavior anyway. Is there a better example to try to illustrate the distinction?

1 Like
3

None of those are opaque closures.

opaque closures can be made by putting @opaque in front of them.

https://github.com/JuliaLang/julia/blob/ca2332e868d24115dd8a75b80fde31eb1f9880fe/base/opaque_closure.jl

3 Likes