# Make immutable mutable again - or make arrays of structures stack-allocated
**URL:** https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800
**Category:** Internals & Design
**Tags:** array, memory-allocation, data\_structures
**Created:** [February 15, 2019, 1:28am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800 "2019-02-15T01:28:34Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@sairus7](https://discourse.julialang.org/u/sairus7)
#### Post date: [February 15, 2019, 1:28am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/1 "2019-02-15T01:28:34Z")
This post is ispired by the following discussions about mutability and stack-allocated arrays: [Initializing an array of mutable objects is surprisingly slow - #6 by cnliao](https://discourse.julialang.org/t/initializing-an-array-of-mutable-objects-is-surprisingly-slow/11780/6)
[Fortran vs Julia stack allocated arrays](https://discourse.julialang.org/t/fortran-vs-julia-stack-allocated-arrays/18119)
[Why mutable structs are allocated on the heap?](https://discourse.julialang.org/t/why-mutable-structs-are-allocated-on-the-heap/12992)
[Mutate NTuple at position](https://discourse.julialang.org/t/mutate-ntuple-at-position/17682)
[Way to have a function "mutate an immutable" without much performance loss](https://discourse.julialang.org/t/way-to-have-a-function-mutate-an-immutable-without-much-performance-loss/1050)
And the actual issue is: [WIP: Make mutating immutables easier](https://github.com/JuliaLang/julia/pull/21912)
It seems to me that many people get confused with mutability, fixed-sized types and stack-allocation, but the point is, we can overwrite whole immutable objects, but not change their parts.
If so, maybe this issue can be resolved by adding another `atomic` type qualifier, along with `mutable` and `immutable`?
```julia
atomic struct Foo
x::Int
y::Int
end
```
The `atomic` treats every field change to be equivalent to the entire structure overwrite. But, knowing that the only one field is changed, it is optimized to only one field overwrite?
This has more consistent syntax with the other functions, since we do not need to have special write syntax, such as [Setfield.jl](https://github.com/jw3126/Setfield.jl) or `MArray` at [StaticArrays.jl](https://github.com/JuliaArrays/StaticArrays.jl), so the same functions can be used for immutables, mutables and atomic types, and we can explicitly define, for which types we want to be able to overwrite individual fields.
---
### Author:  [@jameson](https://discourse.julialang.org/u/jameson)
#### Post date: [February 15, 2019, 1:56am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/2 "2019-02-15T01:56:42Z")
> [@sairus7](#):
>
> The `atomic` treats every field change to be equivalent to the entire structure overwrite
That is currently the definition of `struct`, so that’s currently in theory feasible to add without needing to change anything. The links are typically arguing for a breaking change to (weakening of) that memory model. A more thorough analysis of the approach you mention would be helpful (and eventually PRs to implement the necessary optimizations mentioned “optimized to only one field overwrite”) and the next step to drive those issues forward. It could even be a good GSOC project for someone.
---
### Author:  [@c42f](https://discourse.julialang.org/u/c42f)
#### Post date: [February 15, 2019, 2:17am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/3 "2019-02-15T02:17:04Z")
Making this a property of the type is an intriguing alternative. But it seems to introduce ambiguity into some syntactic constructs which are currently unambiguous:
```julia
atomic struct X
a::Int
b::Int
end
function foo(x)
y = x
# Now the following line is syntactically ambiguous
x.a = 10
# Means one of the following?
setfield!(x, :a, 10) # If `x` is a `mutable struct X`; mutates `y`
x = X(10, x.b) # If `x` is a `atomic struct X`; doesn't mutate `y`
# The value of `y` depends on whether `X` is atomic or not.
return y
end
foo(X(1,1))
```
---
### Author:  [@c42f](https://discourse.julialang.org/u/c42f)
#### Post date: [February 15, 2019, 2:23am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/4 "2019-02-15T02:23:35Z")
So to avoid this ambiguity, wouldn’t it be better to distinguish this special kind of “replace it all” update with actual syntax? Which is what I thought [WIP: Make mutating immutables easier by Keno · Pull Request #21912 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/21912) basically did.
---
### Author:  [@yuyichao](https://discourse.julialang.org/u/yuyichao)
#### Post date: [February 15, 2019, 2:35am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/5 "2019-02-15T02:35:27Z")
As I mentioned many time, syntax with this semantics is totally fine as long as,
1. The syntax should make it clear it’s assigning to the variable, not the field, i.e. `a = a@(b = 2)` is fine, `a@b = 2` or `a.b = 2` are not.
2. This must not be used for `StaticArray` or anything similar. That’s a regression.
---
### Author:  [@tkf](https://discourse.julialang.org/u/tkf)
#### Post date: [February 15, 2019, 2:51am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/6 "2019-02-15T02:51:44Z")
> [@c42f](#):
>
> ```julia
> setfield!(x, :a, 10) # If `x` is a `mutable struct X`; mutates `y`
> x = X(10, x.b) # If `x` is a `atomic struct X`; doesn't mutate `y`
>
> ```
I’ve been wanting to use functions that “_may_ mutate argument but returned value _must_ be used”. For example, in this case I want (say) `setfield!!` which does `setfield!` for `mutable struct` and `Setfield.set` for `struct`. Similarly, I want to use `setindex!!` for writing algorithms that can be used for `Array` and `StaticArray`. Having `push!!` that can be used for both mutable and persistent data structure would also be great.
---
### Author:  [@c42f](https://discourse.julialang.org/u/c42f)
#### Post date: [February 15, 2019, 5:09am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/7 "2019-02-15T05:09:06Z")
> I’ve been wanting to use functions that “ _may_ mutate argument but returned value _must_ be used”
Yes, this makes a lot of sense. I kind of like `setindex!!` but find it hard not to read as “mutate, the argument. No; _really_ mutate it.” 🙂
Did you try using `MArray` recently, by the way? This has really gotten a lot more efficient in julia 1.0 and I’m often surprised by how the compiler is able to avoid allocations completely even with `MArray`. It’s not a complete solution to the problems though.
---
### Author:  [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)
#### Post date: [February 15, 2019, 12:26pm UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/8 "2019-02-15T12:26:27Z")
> [@c42f](#):
>
> I kind of like `setindex!!` but find it hard not to read as “mutate, the argument. No; _really_ mutate it.
`push?!` appears like a fitting name.
Syntax-wise, compared to Keno’s original proposal, we could probably simplify: `a.b.c[d][e].f @= rhs` could mean: Take the rightmost mutable access; that element gets modified and replaced by a new immutable, where the appropriate parts to the right have been replaced by `rhs` and all other fields/indices are unchanged. If there is no mutable thing at all, then we replace `a` with a new object. If the compiler can prove that no other references exist to some immutable non-bitstype that gets replaced, then it can feel free to update the immutable in-place. In other words, there is no reason for the user to specify the point of mutation, like `a.b.c@d.e = rhs`: If we start copy-replacing at any position that is not right-most, then we play havoc with user expectations about what parts are shared after the update, and by refusing to define convenient replace-by-updated sytax for any mutable thing we also protect people from mistaken copies.
If at all possible, it would be very nice if the interplay between `@` and dot-syntax resulted in a way of writing code for `a @= b .+ c` or maybe `a @.= b.+c` that uses the `Base.GMP.MPZ` in-place operations if `a` happens to be `BigInt` and just works for `Int`.
---
### Author:  [@sairus7](https://discourse.julialang.org/u/sairus7)
#### Post date: [February 15, 2019, 1:15pm UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/9 "2019-02-15T13:15:24Z")
> [@c42f](#):
>
> So to avoid this ambiguity, wouldn’t it be better to distinguish this special kind of “replace it all” update with actual syntax?
Ok, so the actual question is, should it be the same syntax with ambiguous behaviour in some specific cases, or should it have different syntax with the need to rewrite every function on it.
Anyway, I will argue that it’s ambiguous. If you define `atomic` type, you expect it to work differently inside functions - you can treat that as some hidden function specialization. Another way is to treat it like a single number, not a complex atomic type.
So, the same function will break with `immutable` and have different behaviour on `atomic` and `mutable` types - these are 3 dirrefent behaviours. For me it’s clear that for `atomic` case you always have a deep copy, and a reference copy for `mutable` case.
Is that sufficient to resolve ambiguities?
By the way, I don’t sure if compiler is optimizing copies on immutable types to just referencing it wihout making the actual copy.
> [@yuyichao](#):
>
> As I mentioned many time, syntax with this semantics is totally fine as long as,
>
> 1. The syntax should make it clear it’s assigning to the variable, not the field, i.e. `a = a@(b = 2)` is fine, `a@b = 2` or `a.b = 2` are not.
> 2. This must not be used for `StaticArray` or anything similar. That’s a regression.
I can provide another example similar to 2: an array of immutable structures. In case of different syntax we should somehow specialize any function in the form:
```
function addone!(a) #specialize for mutables
for i =1:length(a)
a[i].b += 1
end
end
function addone!(a) #specialize for immutables
for i =1:length(a)
a[i]@(b += 1)
end
end
```
This can become inconvenient for anyone who would work with contigious mutable arrays - and this is a pretty frequent case, especially in terms of performance! (see related threads above)
Another solution to have consistent syntax: maybe the compiler should determine if there are no ambiguities (copies of that struct) inside a function and just reinterpret `a[i].b += 1` to `a[i]@(b += 1)`, but in ambiguous cases it should return an error and ask to fix the syntax manually?
---
### Author:  [@tkf](https://discourse.julialang.org/u/tkf)
#### Post date: [February 16, 2019, 2:19am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/10 "2019-02-16T02:19:39Z")
> [@c42f](#):
>
> Did you try using `MArray` recently, by the way? This has really gotten a lot more efficient in julia 1.0 and I’m often surprised by how the compiler is able to avoid allocations completely even with `MArray` .
Interesting! I don’t use `MArray` much so I didn’t know that. But to be honest I kind of like that Julia gives me an “excuse” to do functional programming when performance matters 🙂
---
### Author:  [@tkf](https://discourse.julialang.org/u/tkf)
#### Post date: [February 16, 2019, 2:21am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/11 "2019-02-16T02:21:21Z")
> [@foobar\_lv2](#):
>
> > I kind of like `setindex!!` but find it hard not to read as “mutate, the argument. No; _really_ mutate it.
>
> `push?!` appears like a fitting name.
If `function_name?!` is a valid syntax that would be awesome.
---
### Author:  [@improbable22](https://discourse.julialang.org/u/improbable22)
#### Post date: [February 16, 2019, 8:39am UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/12 "2019-02-16T08:39:18Z")
It’s an error so could in principle be changed, right?
```julia
julia> fun?!(x) = x
ERROR: syntax: space required before "?" operator
julia> fun⁈(x) = x # unicode
ERROR: syntax: invalid character "⁈" near column 5
```
---
### Author:  [@tkf](https://discourse.julialang.org/u/tkf)
#### Post date: [February 16, 2019, 11:34pm UTC](https://discourse.julialang.org/t/make-immutable-mutable-again-or-make-arrays-of-structures-stack-allocated/20800/13 "2019-02-16T23:34:24Z")
> [@improbable22](#):
>
> It’s an error so could in principle be changed
I think so.