How to understand this code that seemingly creates an anonymous type on the fly?

I read this article and found this piece of code that I couldn’t understand how it works. I modified it a bit from the article.

function makeatype(number)
    add() = number += 1
    get() = number
    (var) -> (number; add; get)
end

d = makeatype(5)
d    # #12 (generic function with 1 method)
d.number    # Core.Box(5)
d.get()    # 5
d.add()
d.get()    # 6

I tried to search the Julia’s documentation but couldn’t find anything to help me understand the code. It looks like Core.Box isn’t even in the docs, and it’s undocumented in the REPL.

Some questions:

  • Am I correct that the functions add and get are closures, and the variable number is captured? That allows add and get to access a persistent object/variable number.
  • Is Core.Box(5) the internal representation of the captured variable number?
  • How to understand the last line (var) -> (number; add; get)? I have no clue. Why is the argument var not used / needed? This looks like an anonymous function (and d is indeed is), but how can one access the number field and add and get methods within a function d like in the code? Also, (number; add; get) should return only the last, which is function get, but somehow it looks like a structure with three members number, add, and get.

I’ll appreciate any pointer on how to understand this code. Does this code use some undocumented features of Julia? If that’s the case, is it not a good idea to do this because the features may change in the future?

Yes!

Yeah. It’s really an internal implementation detail and not something you generally need to interact with, although it can be an important thing to pay attention to for performance.

Yeah, this is weird. Clearly the var is unused, and the code works exactly the same way without it:

julia> function makeatype(number)
           add() = number += 1
           get() = number
           () -> (number; add; get)
       end
makeatype (generic function with 1 method)

julia> d = makeatype(5)
#11 (generic function with 1 method)

julia> d.number
Core.Box(5)

julia> d.get()
5

julia> d.add()
6

julia> d.get()
6

So I think the use of var is best explained as mistake or a misunderstanding on the part of the author of that article you read.

At a higher level, this code is relying on an (undocumented?) feature of closures in Julia. When you create a closure like:

julia> function outer(x)
         inner = () -> x
       end

Julia actually creates a new struct type under the hood which looks something like:

struct MyType
  x::<inferred type of `x`>
end

and then it makes that struct callable like a function:

function (t::MyType)()
  return t.x
end

The closed-over variable x gets turned into a field of that newly generated type.

The problem here is that Julia doesn’t make any particular guarantees about whether those fields will actually exist or what type they will actually be, only that calling the closure will somehow work. That’s why you see d.number as the mysterious Core.Box(5). You’re essentially looking into the internal details of a Julia closure, so you’re kind of on your own.

The reason the code in that article does:

() -> (number; add; get)

is just to trick Julia into giving you a closure which happens to contain number add and get as fields because they’re referenced in the (number; add; get) expression.

And, by the way, you’re right to say that:

It just happens that referencing all of those causes the Julia compiler to decide to create fields in the resulting closure. But I don’t think you can or should rely on that.

Basically, the code in that article mostly works (except that number has an unhelpful Box type) on this particular version of Julia, but I wouldn’t necessarily expect it to always work.

Rather than relying on the details of how closures are implemented, we can achieve exactly the same thing in a less confusing way:

julia> function makeatype(number)
           add() = number += 1
           get() = number
           (; add, get)
       end
makeatype (generic function with 1 method)

julia> d = makeatype(5)
(number = 5, add = var"#add#7"(Core.Box(5)), get = var"#get#8"(Core.Box(5)))

julia> d.get()
5

julia> d.add()
6

julia> d.get()
6

In this case, rather than returning a new closure which happens to have fields with the right names, I’m returning a named tuple whose entries are named number, add, and get. The (; add, get) syntax is just a helpful way to make a named tuple:

julia> a = 1
1

julia> b = 2
2

julia> (; a, b)
(a = 1, b = 2)

This avoids the extra anonymous function and the obviously useless var argument.

And, as a final point, I’d probably recommend not writing code like this in “real” Julia code. Using f(x) allows you to participate in Julia’s amazing multiple dispatch system, while x.f() does not.

7 Likes

I think you meant (;number = number, add, get) ?

I still do not understand how/where the value of the number gets “stored” such that subsequent calls to add() increase it.

Is (something like) this what is going on under the hood for the add function?

julia> struct Add
           number
       end

julia> (a::Add)() = a.number[] += 1

julia> add = Add(Ref(5)) # needs boxing if the struct is immutable
Add(Base.RefValue{Int64}(5))

julia> add()
6

julia> add()
7


The code you found uses the same trick shown on StackOverflow by Jeff a few years ago to do some object-oriented programming in Julia: oop - How to create a "single dispatch, object-oriented Class" in julia that behaves like a standard Java Class with public / private fields and methods - Stack Overflow. However note that this isn’t the idiomatic way to write code in Julia

Actually I did that first, but I don’t think it’s really what you want. That captures the value passed into the function, but that value doesn’t change when you call add(). I assumed that’s not actually what the user wants.

Yeah, I think that’s a close enough mental model. You can create the same effect manually with Ref and a let block:

julia> d = let number = Ref(5)
         add() = number[] += 1
         get() = number[]
         (; add, get)
       end
(add = var"#add#9"{Base.RefValue{Int64}}(Base.RefValue{Int64}(5)), get = var"#get#10"{Base.RefValue{Int64}}(Base.RefValue{Int64}(5)))

julia> d.get()
5

julia> d.add()
6

julia> d.get()
6
1 Like

Awesome explanation. Thank you very much!