Say I have an immutable type with some mutable field like:
struct S
imm::Int64
mut::Base.RefValue{Int64}
S(imm, mut) = new(imm, Ref(mut))
end
I wanted to define broadcasting on it so it acts like a 0-dimensional array. The manual Interfaces · The Julia Language suggests
Base.broadcastable(s::S) = Ref(s)
But that’s problematic because the same object gets referenced and changing one changes others:
a = S(0, 0)
b = fill(S(1, 1), 2, 2)
b .= a
a.mut[] = -1
b[begin].mut[] == -1
My workaround was to define:
Base.identity(s::S) = S(s.imm, s.mut[])
Base.broadcastable(s::S) = Ref(Base.identity(s)) # or the non allocating Base.broadcastable(s::S) = Tuple(Base.identity(s))
Redefining identity to anything other than identity(x) = x feels a little wrong. Is this the right approach?
I’ve seen a few discourse threads specific to broadcasting, but are there other things I need to keep in mind when working with immutable types with mutable fields?
This is your problem. fill does not copy, but uses the exact same element. Here’s a snippet from the fill docstring:
help?> fill
search: fill fill! kill foldl findall finally isfile fld filter all
fill(value, dims::Tuple)
fill(value, dims...)
[...]
Every location of the returned array is set to (and is thus ===
to) the value that was passed; this means that if the value is
itself modified, all elements of the filled array will reflect
that modification because they're still that very value. This
is of no concern with fill(1.0, (5,5)) as the value 1.0 is
immutable and cannot itself be modified, but can be unexpected
with mutable values like — most commonly — arrays. For example,
fill([], 3) places the very same empty array in all three
locations of the returned vector:
julia> v = fill([], 3)
3-element Vector{Vector{Any}}:
[]
[]
[]
julia> v[1] === v[2] === v[3]
true
julia> value = v[1]
Any[]
julia> push!(value, 867_5309)
1-element Vector{Any}:
8675309
julia> v
3-element Vector{Vector{Any}}:
[8675309]
[8675309]
[8675309]
To create an array of many independent inner arrays, use a
comprehension instead. This creates a new and distinct array on
each iteration of the loop:
julia> v2 = [[] for _ in 1:3]
3-element Vector{Vector{Any}}:
[]
[]
[]
julia> v2[1] === v2[2] === v2[3]
false
julia> push!(v2[1], 8675309)
1-element Vector{Any}:
8675309
julia> v2
3-element Vector{Vector{Any}}:
[8675309]
[]
[]
[...]
It also has nothing to do with .=. You’d also get the same result by writing a loop, i.e.
for i in eachindex(b)
b[i] = a
end
the problem is just that you’re assigning the same === value to each index. This is pretty fundamental and not easy to bypass (even if you did change the broadcast machinery, you’d run into this sort of thing).
You just need to instead make sure you add new values to each element of the array. E.g. a kinda cute way you could write this is
b .= S.(0, 0)
the . in the S.(0, 0) means that you’ll get a fresh application of S for each broadcast iteration.
Do think hard about whether you actually need the field to be mutable. Mutables are useful when you want the “set once and change everywhere” kind of behavior, which it seems you might not.
It’s almost free to construct a new immutable with an altered field (and Accessors.jl makes this convenient) to simply “overwrite” an old one, and often this is cheaper than mutating a mutable.
But if you truly need the mutable field, then others here have provided good feedback on doing so.
As explained already, assignment does not copy in Julia. Instead of changing broadcasting to implicitly copy, it’s probably better to not break the default behaviour, but just copy explicitly:
Base.broadcastable(s::S) = Ref(s)
Base.copy(s::S) = S(s.imm, s.mut[]) # Your identity which was a copy already
b .= copy.(a) # Creates many copies unlike b .= copy(a)
Not necessarily. Maybe if it’s as simple as new putting fields together, and the compiler might elide even that. However, constructors often take time to process inputs. Field mutation goes through setproperty! or similar methods instead of constructors, and it’s much easier to dodge the input processing via Core.setfield!.
julia> struct MyInt
inner::Int
function MyInt(x::Int)
sleep(2) # input validation, conversion, errors, etc
new(x)
end
end
julia> @time x = MyInt(1)
2.008967 seconds (4 allocations: 112 bytes)
MyInt(1)
julia> using Accessors
julia> @time @reset x.inner = 2
2.010840 seconds (5 allocations: 128 bytes)
MyInt(2)
There used to be a Expr(:new,...) hack to get around constructors, but I haven’t seen it exposed in any way. It makes sense because we’re not supposed to dodge input validation and make wrong instances, mutable or not. We can’t even have a lower Core level of setindex! to dodge anything. If we want to trim a composite type’s input validation during number crunching, it’s usually more reasonable to extract the fields individually and put them back in a composite type only when we want to check.
Mutable types either don’t have validation, do validate and also validate any change to their state, or are subject to being hilariously broken by being mutated to invalid states (or using Core.setfield! to skip any validation). You aren’t comparing apples to apples if you’re making invalid or unchecked mutations to a mutable and comparing that to checked constructions of an immutable.
Sometimes the compiler does struggle and doesn’t change an immutable overwrite to a mutation (which the compiler is allowed to do if the old data is no longer used), but this mostly happens for large or very complicated structs.
Thinking more about it, I suppose b .= S.(0, 0) works because Integers behave like zero-dimensional arrays for the sake of broadcasting. Otherwise you’d need something like b .= S.(Ref(0), Ref(0)).
Now that I’m squinting at the docstrings, they’re not asserting that ones or zerosmust fill an Array with the same instance, let alone instantiated once by one or zero, that just happens to be the Base implementation. Could also be worth clarifying whether extending it must be done via one or zero.