How to design with immutable and also mutable type?

[tomtom]

suppose I have a parametric type as follows:

struct A{T <: AbstractVector{Float32} }
    data::T
end

setdata!(a::A, x) = (a.data .= x)

here T could be Vector{Float32} or a view etc.

however, I would like to use SVector (in StaticArrays) also. But as I need to mutate the contents of data over time, I need to define another struct which is mutable like:

mutable struct B{T <: AbstractVector{Float32} }
    data::T
end

setdata!(b::B, x) = (b.data = x)

so, my question is: is there a more beautiful way to do this? maintaining a immutable type along with another mutable type seems ugly… and the behaviors of A and B are basically identical, except for setdata!().

another question is: if I just use (the mutable) B for all cases, would it be stupid because of the performance penalty caused by mutable types?

thanks.

[Zach_Christensen]

Look at ArrayInterface.jl on detecting mutability. I’ll write an example once I’m not on a phone.

[pixel27]

Can I ask why you don’t want to make A mutable? I mean I guess if data is a Vector{Float32}, at a low level you will have 3(?) pointers to get at the data instead of 2. Not sure what the CPU overhead on that would be, but it should be minuscule.

That said, you might be able to do something like:

struct A{T}
    data::T
end

mutable struct B{T}
    data::T
end

A(d::T) where T <: SVector = A(B(d))

getdata(a::A{T}) where T = a.data
getdata(a::A{T}) where T <: B = a.data.data

Granted any changes you want to make to the data will need to use the getdata function.

[tomtom]

thanks. looking forward to your examples.

[tomtom]

@pixel27

Can I ask why you don’t want to make A mutable?

I assume that a mutable struct is much slower than a immutable one, but I’m not sure.

A(B(d)) not work. A and B have many (identical) fields other than data (not shown for simplicity).

[Zach_Christensen]

Wait, do you have some instances where you don’t need to update data and that’s why you have two types in the OP? Or will you for sure need to change the data for every instance of A?

If you will always need to change values within data then you don’t want SVector. You may want to use MVector or just a Vector if you’re going to change sizes.

[tomtom]

@Zach_Christensen

will you for sure need to change the data for every instance of A ?

yes sure, I need to change data for every instance.

@Zach_Christensen

You may want to use MVector

I don’t want to use MVector because:

1. it’s much slower than SVector
2. everytime I’m changing all contents of data, not at a particular index. Effectively I will put a new SVector into data

[Zach_Christensen]

I think I get it. So you want to be able to swap out the SVector but you also want the type to not be mutable because all the other fields stay the same and don’t change.

I think what you want to do is something more like:

struct TypeWithManyFields end

struct SVectorAndFields
    fields::TypeWithManyFields
    data::SVector
end

Then make a new instance of SVectorAndFields like so:

old_data = SVectorAndFields(TypeWithManyFields(), SVector(1))

new_data = SVectorAndFields(new_data.fields, SVector(2))

[tomtom]

@Zach_Christensen

because all the other fields stay the same and don’t change.

no… it’s not the point…

let me try to clarify:

1. I want to have a custom type containing a data field which can be a mutable vector (e.g. Vector) or an immutable vector (e.g. SVector).
2. the whole content of data has to be updated over time. For mutable vector, it could be done by obj.data .= v, i.e. the contents of data are overwritten by those in v. However, for immutable vector, because its content could not be modified, the field data has to be replaced like obj.data = v.
3. now, for obj.data .= v we can use a normal (immutable) struct. But to accommodate obj.data = v, we need a mutable struct instead.

the problem is how to get the best from both worlds (i.e. using a fast, immutable struct for mutable vector & using a mutable struct for immutable vector) without defining two custom types.

or, maybe two types are indeed needed, then the question becomes how to abstract / factor these two types into a single type as their behaviors are basically identical.

[tkf]

FYI, GitHub - JuliaFolds/BangBang.jl: Immutables as mutables, mutables as immutables. is aiming at providing a unified API for immutable and mutable objects. In particular, something like BangBang.@set!! x.data[1] = 0 should work for x :: A{<:Vector} and x :: B{<:SVector}. It even works for x :: A{<:SVector} so you don’t need to define the struct B. This “magic” is implemented in https://github.com/jw3126/Setfield.jl

[Zach_Christensen]

@tomtom

the problem is how to get the best from both worlds (i.e. using a fast, immutable struct for mutable vector & using a mutable struct for immutable vector) without defining two custom types .

I’m still not sure I understand exactly what you’re trying to do. If you want mutable fields then this will allow you to have mutable fields with SVector.

mutable struct MyFields
    field1
    field2
    field3
end

struct MyVectorFields
    fields::TypeWithManyFields
    data::SVector
end

You said that data will need to be changed for every instance and the fields are going to change too. If they are all changing at the same time then just create a new instance every time they are all changed. In which case this would be fine:

struct MyVectorFields
    field1
    field2
    field3
    data::SVector
end

If you’re concerned with the first example because you are changing data a lot more frequently than the rest of your fields and don’t want the overhead of constructing a new instance (which should be small since it’s only two fields), then data probably shouldn’t be part of the rest of the data.

It’s difficult to know exactly where the problem lies with any of these solutions without a more concrete example of what you are trying to do.

[tomtom]

inspired by your suggestions, I realized that the solution is quite simple: instead of calling

setdata!(obj, x)

I should call

obj = setdata!(obj, x)

that’s it! (despite that this calling convention is not idiotmatc)

### normal & fast immutable struct
struct A{T <: AbstractVector}
    data::T
    others::Int
end

### traits
abstract type IsMutableTrait end
struct Mutable <: IsMutableTrait end
struct Immutable <: IsMutableTrait end
ismutable(::Type{<:SVector}) = Immutable()
ismutable(::Type{<:AbstractVector}) = Mutable()

function setdata!(obj::A{T}, x::T)::A{T} where {T}
    setdata!(ismutable(T), obj, x)
end

function setdata!(::Mutable, obj::A{T}, x::T)::A{T} where {T}
    obj.data .= x
    return obj    # *** returns itself ***
end

function setdata!(::Immutable, obj::A{T}, x::T)::A{T} where {T}
    newobj = A(x, obj.others)
    return newobj    # *** returns a new object ***
end

v = A([1, 2], 0)
sv = A(SVector(1, 2), 0)

julia> v = setdata!(v, [3, 4])
A{Array{Int64,1}}([3, 4], 0)

julia> sv = setdata!(sv, SVector(3, 4) )
A{SArray{Tuple{2},Int64,1,2}}([3, 4], 0)

in this way, I got the following advantages:

1. only a single (fast, normal, immutable) struct is needed; no need to maintain two structs
2. updating a mutable data could be done in-place (by .=)
3. a new object is needed to be created only during updates of the immutable data

[tkf]

I think what you just came up with is a nice API. That’s how I’d do if I were to do this in a self-contained module.

It’s just a nit-picking, but I think the API

@tomtom

instead of calling

setdata!(obj, x)

I should call

obj = setdata!(obj, x)

is somewhat non-conventional. Usually in Julia, for API f!(dest, args...), I think you’d expect dest to be guaranteed to be mutated. However, your setdata! sometimes don’t. I think it’s OK for an internal function. Base has something like this; e.g., grow_to!. But if you are going to expose it as a public API, maybe it’s confusing?

(BTW, that’s why I’m using a strange !! suffix in BangBang.jl. But I don’t think it’s a conventional suffix.)

[Tamas_Papp]

@tkf

I think you’d expect dest to be guaranteed to be mutated.

I wouldn’t. I would expect that it allows dest to be mutated, but that’s an option.

@tkf

that’s why I’m using a strange !! suffix

FWIW, I would be fine with just ! for this case.

I don’t think that it is common for callers to depend on the contents of an argument being actually mutated. Consider eg

using LinearAlgebra
A = Matrix(Diagonal(ones(2)))
B = cholesky(A)
rdiv!(A, B)

A conforming program couldn’t even tell if A was mutated here, since it is == to the original.

[tomtom]

@tkf

is somewhat non-conventional.

yeah… it’s not idiomatic. And it’s the very reason I spent so much time stuck in the problem!!!

maybe I should better rename setdata!() as something like setdataΔ() to remind myself (and others) of this strange usage.

just for completeness:

### normal & fast immutable struct
struct A{T <: AbstractVector}
    data::T
    others::Int
end

### traits
abstract type IsMutableTrait end
struct Mutable <: IsMutableTrait end
struct Immutable <: IsMutableTrait end
ismutable(::Type{<:SVector}) = Immutable()
ismutable(::Type{<:AbstractVector}) = Mutable()

function setdataΔ(obj::A{T}, x::T)::A{T} where {T}
    setdataΔ(ismutable(T), obj, x)
end

function setdataΔ(::Mutable, obj::A{T}, x::T)::A{T} where {T}
    obj.data .= x
    return obj
end

function setdataΔ(::Immutable, obj::A{T}, x::T)::A{T} where {T}
    newobj = A(x, obj.others)
    return newobj
end

v = A([1, 2], 0)
sv = A(SVector(1, 2), 0)

v = setdataΔ(v, [3, 4])
sv = setdataΔ(sv, SVector(3, 4) )

julia> v = setdataΔ(v, [3, 4])
A{Array{Int64,1}}([3, 4], 0)

julia> sv = setdataΔ(sv, SVector(3, 4) )
A{SArray{Tuple{2},Int64,1,2}}([3, 4], 0)

[tkf]

@Tamas_Papp

I don’t think that it is common for callers to depend on the contents of an argument being actually mutated.

Functions like pop!, splice!, take!(channel), put!, etc. depend on mutation.

I’m not sure what you want to point out with rdiv! example. It’s docstring says

Compute A / B in-place and overwriting A to store the result.

I’d argue that rdiv! method implementation that does not mutate A has a bug.

[tkf]

@Tamas_Papp

I think you’d expect dest to be guaranteed to be mutated.

I wouldn’t. I would expect that it allows dest to be mutated, but that’s an option.

So do you think it’s OK to add APIs like push!(::Tuple, _) and delete!(::NamedTuple, ::Symbol) in Base? What about push!(::StaticVector, _)? I personally find them strange but if I’m a minority I’m happy to use such APIs as they are very useful.

[Tamas_Papp]

@tkf

I’d argue that rdiv! method implementation that does not mutate A has a bug.

Consider the hypothetical

my_rdiv!(A::AbstractMatrix, J::UniformScaling) = my_rdiv!(A, J.λ)

function my_rdiv!(A::AbstractMatrix, λ::Real)
    λ == 1 ? A : LinearAlgebra.rdiv!(A, λ)
end

It may or may not mutate A. I am not proposing it as something elegant or efficient, but I think it would be conforming.

@tkf

So do you think it’s OK to add APIs like push!(::Tuple, _)

No. The method should do what the contract prescribes, and I would expect that after

push!(collection, item)
@assert item ∈ collection

holds.

My point is very simple: all I am saying that if the method actually does what it promises, whether it mutates a given argument is irrelevant (as long as that wasn’t promised). Eg if the recommended way to use a method is

result2 = buffered_calculation!(result, arguments)

then the method could have result === result2 and overwrite it (eg for Array) or make a new one (SArray). As long as this is only conditional on the type, the compiler can deal with it just fine.

[tkf]

So you are just saying that when “do nothing” is the correct answer, the caller should expect the argument is not mutated (e.g., sort!([1, 2, 3])). I agree, of course. I guess you missed my initial point. It was that it’s not possible to define (say) push! semantics on immutable corrections like tuples; hence the weird suffix.

[Tamas_Papp]

To be fair, I think you are missing my point too: all I am saying is that my interpretation is that ! allows a specific argument to be mutated, but does not require it per se.

For push!, the requirement for mutating the collection comes from the fact that it should contain item.