Mutable struct pitfall

General Usage
1 
julia> mutable struct S x::Int end
julia> a = S(3)
S(3)
julia> b = a
S(3)
julia> b.x = 5
5
julia> a
S(5)

Where is it documented ? Every computer scientist is aware of the risks when writing b = a, and then modifiying elements of b, when a is an array; but seriously who knows that the pb is the same with mutable struct ?

You don’t imagine how much time I spent to discover and understand very subtle anomalies in a somewhat complex program, where some variables were cached in dictionaries, that evolved mysteriously because some of them were these evil mutable struct…

2

As you noted, when doing a = b, a and b reference the same object (mutable or not, this is always true)

Therefore, the effect you observed is true for anything mutable in julia. Dicts, Arrays, Sets, etc… since a is b, mutating one is mutating the other.

6 Likes
3

Can you give examples of popular languages where this isn’t what happens?

Your example seems like it’s what happens with Python’s classes, for example:

>>> class Foo:
...     def __init__(self, x):
...         self.x = x
...
>>> a = Foo(1)
>>> b = a
>>>
>>> a.x
1
>>> b.x
1
>>>
>>> b.x = 2
>>>
>>> a.x
2
>>> b.x
2
8 Likes
4

So the simple solution is: Don’t use evil mutable structs. Use immutable structs instead.

2 Likes
5

Since @johnmyleswhite replied, maybe it’s a good moment to refer to this old-but-still-very-relevant post of his:

3 Likes
6

?=:

= is the assignment operator.

  • For variable a and expression b, a = b makes a refer to the value of b.

Almost all Julia users I assume, since it is the same semantics for all values. After a = b, no valid program should be able to distinguish a from b, ie a === b.

Incidentally, I wish we had evil as a syntax qualifier, as in

evil mutable struct S
    x::Int
end

evil function foo(x::Int)
    x + 1
end

I am still not quite sure what it would do though, but we could think of something.

6 Likes
7

I didn’t see the keyword struct outside the C language, and in this case assignment is a copy.

1 Like
8

In julia it is the same. That is, assignment of immutable struct is a copy. mutable struct means “pointer to heap-allocated struct with C layout, but everything managed by the julia garbage collector + allocator please”.

Structure layout is julia tries to be mostly C compatible (in-order, same alignment rules, …). Mutable structs, as “proper reified objects”, have an identity, a type-tag and a memory address (except if the compiler figures out that it can get away without assigning an address because nobody uses it…).

1 Like
9

No, no, no. It works semantically exactly the same as for mutable structures. The rest is compiler optimizations.

5 Likes
10

Fair enough. I think Java has similar behavior but they use the keyword class instead. If you want to think in terms of C structs, maybe the most sane analogy is that the problem is not the struct itself (mutable or immutable), but that all variables in Julia are NOT names of positions of memory, but instead, names of pointers to the memory. So, if your swap two variables i.e., a, b = b, a you just swap pointers instead (often you do not even pay the price of a pointer swap, the compiler elides this).

This is the best I can do using concepts of C, the truly best would be learning how Julia has bindings and not variables. @yuyichao has been answering this question for years. If they give a reference (maybe to a post of them), I will always link it whenever I see a variation of this question.

1 Like
11

Oh that’s a delightful idea. Pretty easy to cook up at least on the documentation level as well:

julia> macro evil(ex)
           quote
               """
                   $($(repr(ex))) 
        
               This is evil, avoid it!
               """
               $ex
           end |> esc
       end
@evil (macro with 1 method)

julia> @evil mutable struct S
           x::Int
       end
S

help?> S
search: S Sys Set SVD sum sin sec Some svd sum! step stat sqrt sort skip size sinh sind sinc sign show seek sech secd svd! Schur

  :(mutable struct S
    #= REPL[1]:2 =#
    x::Int

  end) 

  This is evil, avoid it!

julia> @evil function foo(x::Int) = x + 1
ERROR: syntax: unexpected "="
Stacktrace:
 [1] top-level scope at none:1

julia> @evil foo(x::Int) = x + 1
foo

help?> foo
search: foo floor pointer_from_objref OverflowError RoundFromZero unsafe_copyto! functionloc StackOverflowError @functionloc

  :(foo(x::Int) = begin
        #= REPL[4]:1 =#
        x + 1
    end)

  This is evil, avoid it!
2 Likes
12

Since you asked, FWIW, struct assignment in Go is a copy. I suppose C and C++ are still popular and they behave the same. You would assign pointers (or references in C++) to have two variables reference the same instance.

1 Like
13

Isn’t an array a mutable struct?

2 Likes
14

I think that their “struct assignment” is not Julia’s =, it is closer in spirit to something like .=.

I think the misunderstanding in this topic comes from some languages allowing sophisticated semantics for a = b, with eg C++ allowing full overloading so that literally anything can happen, including side effects etc.

In contrast, Julia’s = is simple: all a = b does is ensure a === b — no tricks, just a few corner cases like type declarations.

example 
julia> function f(x)
           local y::Float64
           y = x
           x ≡ y
       end
f (generic function with 1 method)

julia> f(1)
false

julia> f(1.0)
true
2 Likes
15

Julia language has a beautiful design, and is quick. That’s why I use it, without understanding (ot trying to understand) its internal structure. I think (and hope) I’m not the only one in this situation. Something like :

the truly best would be learning how Julia has bindings and not variables. @yuyichao has been answering this question for years

isn’t for me – I tried to read it, unsuccessfully :frowning:

Why there are both struct and mutable struct was obviously surprising, so I read the doc : https://docs.julialang.org/en/v1/manual/types/#Mutable-Composite-Types-1. It’s well written and interesting, but doesn’t warn users about the pitfall that’s the subject of this conversation.

Sure using two names for an array, a dictionary, and so on, is hazardous, but any programmer with some experience knows that. At the opposite any programmer with some experience is aware of the composite data type struct when using the C language, that’s why I think Julia documentation should be completed.

Have a nice sunday !

16

Since you just encountered this problem, you still remember why you found it confusing, so it would be great if you could contribute to the docs.

I suspect that the Types section may not be the best place for this, since it is really general to assignment. Note that the Noteworthy differences from C/C++ has

Julia arrays are not copied when assigned to another variable. After A = B , changing elements of B will modify A as well. Updating operators like += do not operate in-place, they are equivalent to A = A + B which rebinds the left-hand side to the result of the right-hand side expression.

and

Julia values are not copied when assigned or passed to a function. If a function modifies an array, the changes will be visible in the caller.

Perhaps these could be reworded from “arrays” to “containers (including arrays and mutable structs)”.

3 Likes
17

Yes, that’s all true. I’m not making a case for or against how Julia works, just answering the question. Every language is different. One of the most important features to understand when learning a new language is its uses of value vs reference semantics.

2 Likes
18

Julia makes that easy. There are no copy constructor or move constructors etc. Assignment is always a no-op. It always just binds the symbol to the object on the right hand side. No implicit copies. Same when passing arguments to functions.

3 Likes