How to change arrays of a mutable struct

New to Julia
1

Hey there, I am currently trying to make a Dataset struct object that has new X and y values to it appended to it over time. This is my set up

mutable struct Dataset{T<:Real}
    X::AbstractMatrix{T}
    y::AbstractVector{T}
end


function Dataset(
        X::AbstractMatrix{T},
        y::AbstractVector{T}
	) where {T<:Real}
    return Dataset{T}(X, y)

end

function extendDataset(dataset::Dataset, X::AbstractMatrix{T}, y::AbstractVector{T}) where {T <: Real}
	catX = hcat(dataset.X, X)
	caty = vcat(dataset.y, y)
    dataset.X = catX
    dataset.y = caty
    dataset.X = 2 * dataset.X
    dataset.y = 2 * dataset.y
end

All 4 of these assignments, even the ones that simply do a 2 * dataset.X and 2 * dataset.y give me the following error whenever I call extendDataset:

MethodError: convert(::Type{Union{}}, ::Array{Float32,1}) is ambiguous. Candidates:
  convert(::Type{Union{}}, x) in Base at essentials.jl:169
  convert(::Type{T}, a::AbstractArray) where T<:Array in Base at array.jl:554
  convert(::Type{T}, arg) where T<:VecElement in Base at baseext.jl:8
  convert(T::Type{var"#s91"} where var"#s91"<:BitArray, a::AbstractArray) in Base at bitarray.jl:574
  convert(T::Type{var"#s828"} where var"#s828"<:SparseArrays.SparseVector, m::AbstractArray{T,1} where T) in SparseArrays at /home/dhananjay/julia-1.5.3/share/julia/stdlib/v1.5/SparseArrays/src/sparsevector.jl:434
  convert(T::Type{var"#s828"} where var"#s828"<:SharedArrays.SharedArray, a::Array) in SharedArrays at /home/dhananjay/julia-1.5.3/share/julia/stdlib/v1.5/SharedArrays/src/SharedArrays.jl:369
  convert(::Type{SA}, a::AbstractArray) where SA<:(StaticArrays.SArray{Tuple{},T,0,1} where T) in StaticArrays at /home/dhananjay/.julia/packages/StaticArrays/NTbHj/src/Scalar.jl:12
Possible fix, define
  convert(::Type{Union{}}, ::Array{T,1} where T)
convert at ./some.jl:34

Does anyone know what I am doing wrong?

2

It should depend on what arguments you’re calling it with, but I also think you want DataSet{T} in the function signature.

3

By the way, if you append! and push! to your arrays instead of replacing them, you don’t need a mutable struct. And it’s better for performance to have concrete types than abstract ones in your struct definition.

4

Can you post a complete example that produces the error? The following works in Julia 1.6-rc1:

mutable struct Dataset{T<:Real}
    X::AbstractMatrix{T}
    y::AbstractVector{T}
end

function Dataset(
        X::AbstractMatrix{T},
        y::AbstractVector{T}
	) where {T<:Real}
    return Dataset{T}(X, y)
end

function extendDataset(dataset::Dataset, X::AbstractMatrix{T}, y::AbstractVector{T}) where {T <: Real}
	catX = hcat(dataset.X, X)
	caty = vcat(dataset.y, y)
    dataset.X = catX
    dataset.y = caty
    dataset.X = 2 * dataset.X
    dataset.y = 2 * dataset.y
end

julia> d = Dataset([1 2; 3 4], [1,2])
Dataset{Int64}([1 2; 3 4], [1, 2])

julia> extendDataset(d, [4 6]', [3])
3-element Vector{Int64}:
 2
 4
 6

However, why define a Dataset constructor? It seems equivalent to the default constructor. I mean that the following works:

mutable struct Dataset{T<:Real}
    X::AbstractMatrix{T}
    y::AbstractVector{T}
end

julia> Dataset([1 2; 3 4], [1,2])
Dataset{Int64}([1 2; 3 4], [1, 2])

For the function signature I would just write extendDataset(dataset::Dataset, X, y) which has the same performance but is more general (it works for other types, as long as they can be converted).

However I agree with @gustaphe: if you only need to grow your matrix and vector, it’s better to use an immutable struct. A mutable struct might be the right choice if you really need to replace X or y for some reason.

1 Like
5

Hey there. Thank you for your reply: so I tried it out and it seems like the code works in REPL but when I use it in my larger project it does not work: the reason why I have a very redundant looking dataset constructor is because this is actually a part of a larger object. To reproduce the error I am getting, you could clone and run python example.py on : https://github.com/DhananjayAshok/PySR/tree/lgga with https://github.com/DhananjayAshok/SymbolicRegression.jl/tree/LGGA

However as per your question on mutable vs immutable structs, I originally tried to do this without redefiining X and y but while I can append or push to y which is array when I try to append or push to a Array{T, 2} it gives me an error. If I had
a = rand(3, 100) b = rand(3, 10) append!(a, b) push!(a, b)
neither of those work, is there a way to in place extend a with b?

I have also tried cat but it seems cat! does not exist so how can I do this?

6

I’m surprised to find you’re right about append!. You could consider storing X as an Array of Arrays, or use something like ElasticArrays.

1 Like
7

I also didn’t know these methods only work on 1D arrays! See Resize!(matrix) for a discussion.

@DhananjayAshok can you try to make a minimal working example in Julia that reproduces the problem? Or at least show the exact error message that you get?

8

Hey everyone. Thanks again for leaving your replies on this. I did some more testing and realised something weird was happening. I only get this error when running it as a multiprocessing task, when I run it normally it works. This confuses me but not exactly the question that I originally asked so I shall close the issue.