Mutable struct vs closure

The main disadvantage I see with mutable struct is that its state can change, so it makes code more complicated to reason about, and less safe. This closure approach does not seem to address that, in fact it makes it easy to create bugs like this:

# a function intended to return a counter starting at 100
const COUNTER_100 = Counter(100)
get_counter_100() = COUNTER_100

# testing the above function
c = get_counter_100()
c.get()   # returns 100
c.add1()
c.get()   # returns 101

# testing it again
d = get_counter_100()
d.get()   # oops! returns 101

Furthermore, the closure approach may lead to code like this:

(get, add1, minus1) = Counter(3)

which is not good. Whenever you change the signature of Counter in the future, e.g. to return more functions, you’ll need to find and update every line like this.

As others have pointed out, the closure approach is also harder to work with, less readable, and less efficient. So overall, I don’t really see any advantages of using it, and I would go with either an immutable struct just like in @dpsanders post above (creating a new instance for each modification) if you want the advantages of immutability, or simply a mutable struct.

I’m curious though where you got the idea, have you worked with languages/frameworks where this particular closure-based pattern is preferred? Anything you can link to?

3 Likes

haha, we’re all “old” guys in this forum I guess. I used to code in C, C++, Java, R…

as @WschW has mentioned, traditional OO supports “private” fields, and methods “belong” to a “class”. The type system in julia is a new concept for me… I still feel “dangerous” using the mutable struct way…

this does not work well if the type is composite. For example:

struct Composite
    smallest::Int64
    largest::Int64
    values::Vector{Int64}

    function Composite(x::Vector{Int64})
        new(minimum(x), maximum(x), x)
    end
end

function newelement(c::Composite, x::Int64)::Composite
    return Composite([c.values; x])
end

julia> a = Composite([2, 3, 5])
Composite(2, 5, [2, 3, 5])

julia> b = newelement(a, 1)
Composite(1, 5, [2, 3, 5, 1])

as the composite type contains a non-scalar field (i.e., values), every new Instantiation of this immutable type would result in memory allocation

… which may or may not be the bottleneck in particular code, so worrying about it prematurely does not always pay off.

Also, if you happen to be accumulating an ex ante unknown number of elements, you cannot always avoid some memory allocation.

The best approach is to design a small interface, which provides a layer over the actual mechanism. You can benchmark and optimize the latter, and change it without affecting the rest of the code.

2 Likes

I think, it’s worth discussing and thinking deeply on the tradeoffs between the flexibility of mutable struct and the benefits of traditional encapsulation.

Possibly, but Julia does not offer what you call traditional encapsulation, a mechanism designed to explicitly hide state (or make it difficult to access).

Instead, I try to stick to the following implicit rules:

  1. only methods of the same module should modify fields of a structure,

  2. reading fields is kind of borderline, if it is needed to done frequently, it should be done with an accessor, or documented as the recommended API.

If you are really paranoid, you could redefine

Base.getproperty(::MyObject, _) = error("no access to fields")

or similar. But you always have Base.getfield to circumvent this. There is really no way to protect people if they want to do something stupid.

2 Likes

Very interesting question!
I often encapsulate my states like the following:

function _add(x, c)
    new_x = x + c
    new_c -> _add(new_x, new_c), new_x
end
function init_add(x)
    c -> _add(x, c)
end
add = init_add(2)
add, a = add(3) # a == 5
add, a = add(4) # a == 9

Every _add function prepares and returns a new _add function with the current states initialized.
This does not have the problem of Core.Box, does it?
And the good thing is that the user does not have to care about the variables inside.
What do you think of this approach?

It’s a good idea. And it would be nice to be enforced in the language syntax, I think.

hm… an interesting approach. Could you tell me what’s the name of this approach?

anyway, it seems odd to me that every time we call add(), it returns a new one and effectively removes the old one…

Just like for loops are not first class?

The lowering doesn’t define the language. It is an implementation of the language.

3 Likes

You may be interested in the discussion here:

but this is probably something a linter could also do.

1 Like

I made this approach up myself. I don’t know if it has a name. It works fairly well for my implementations for e.g.:

time_update = init_kalman_filter(x, P)
measurement_update = time_update(F, Q)
time_update, x, P = measurement_update(H, R)
#...

works like a charm

Another trick I find amusing to make it harder to access fields is using field names that start with the comment character “#”. In your case this is how I’d do it:

julia> @eval mutable struct Counter
           $(Symbol("#count"))::Int
       end

julia> @eval function add1!(x::Counter)
           x.$(Symbol("#count")) += 1
       end
add1! (generic function with 1 method)

julia> @eval function minus1!(x::Counter)
           x.$(Symbol("#count")) -= 1
       end
minus1! (generic function with 1 method)

julia> @eval Base.getindex(x::Counter) = x.$(Symbol("#count"))

julia> c = Counter(3)
Counter(3)

julia> add1!(c)
4

julia> c[]
4

julia> minus1!(c)
3

julia> c[]
3

julia> c.#count = 2


ERROR: syntax: incomplete: premature end of input

julia> c[] = 2
ERROR: MethodError: no method matching setindex!(::Counter, ::Int64)
Stacktrace:
 [1] top-level scope at none:0

1 Like

That’s all rather convoluted just to make a piece of data private, and you can still access it anyway

julia> getfield(c,Symbol("#count"))
3

It sounds like what people really want is the ability to make something private in a certain scope.

For example, it would be great to declare a private variable in the module instead of defining a struct or closure for it.

Yes, in Julia, at the moment, there is no such thing as an unaccessible field. This is just one more trick among others to hide from public API how to change the value of the struct.

So it is really just a matter of making it harder to accidentally change the field. If one is savvy enough to find out how to do that, then, supposedly, one knows what he is doing…

It seems even C++ class private members can be accessed, if someone really wants to.

2 Likes

Yes. Marking something as “private” in languages like julia or C++ is just a documentation hint for your downstream. Dedicated users will point a disassembler at your binary and just poke at the memory. Then, ten years down the line, some poor sod will be stuck supporting backwards compatibility for these hacks.

If you want to enforce encapsulation, then you must run on a virtual machine that does this for you, like the JVM. There is a plethora of languages targeting the JVM besides java.

Julia targets the hardware, without an intermediate enforcement layer. Since dedicated users will access your internal fields anyway, what is the additional gain above clearly documenting that the field is internal, both by actual docs and by using suggestive names?

I’m not sure what the OP had in mind, but this reminds me of Let Over Lambda by Doug Hoyte. One of the patterns advertized in the book is called let over two lambdas, and illustrates the parallel between closures and objects. It looks like this (in Common Lisp):

(let ((counter 0))
  (values
    (lambda () (incf counter))
    (lambda () (decf counter))))
1 Like

It depends what you want to do. If you just want to update in place, as Simon said, do

import Base: push!

push!(x::Composite, i) = push!(x.values, i)

(You will also need to update smallest and largest.)

Note that even though Composite is an immutable struct, mutable objects inside the struct are still mutable.

Also, if you make a “copy constructor” as

julia> Composite(x::Composite) = Composite(x.values)

then you do not allocate the vector, but rather reuse it:

julia> push!(x.values, 10)
3-element Array{Int64,1}:
  1
  2
 10

julia> y
Composite(1, 2, [1, 2, 10])

i.e. y has also changed.

If you don’t want this then indeed you have to allocate a new vector (e.g. with copy).

2 Likes

Can someone explain to me what kind of safety is actually at issue? Is the concern malicious code, or accidentally doing something you didn’t mean to do (or a fear users will accidentally do something unintended)?

If the latter, it seems like @Tamas_Papp’s point about a clear API is ideal - it took me a while to figure out when I first started with Julia, but it’s been a long time since I accessed a field directly (I did it because the accessor didn’t exist, and the package maintainers responded to an issue and added one).

1 Like

I guess the usual concern is a large team of programmers, where some members could do something quick & dirty by exposing internals, which could then lead to a bug. Internal style guides and code review protect against this to some extent, and as others have pointed out, there is no protection against a sufficiently determined individual who is bent on shooting themselves in the foot.

But some languages, eg C++, offer facilities for this that at least raise the cost of accessing internals, and some people miss them in Julia.

2 Likes