Automatic resizing of struct internal array when value changes?

[Ahmed_Salih]

Hello!

Suppose I have a struct as such:

mutable struct TheStruct
      N::Int
      x::AbstractVector{Float32}
end

And at different times “N” might change, for example from 5 to 7. When this happens I want to resize!(x,N) using the updated value of N = 7.

Is there a way to do this automatically, such there is some kind of “listener”?

Kind regards

[Sukera]

Not in general, no. You can overload setproperty! to also resize the array when N changes, but that won’t prevent a determined user from bypassing your invariant using setfield!.

[Ahmed_Salih]

That sounds good to me, I am doing this mostly to learn and give it a shot! I think it could be useful for me.

Would you happen to know how to write such an overload of setproperty!?

Kind regards

[Sukera]

Both getproperty and setproperty! are kind of fiddly to get right, since they inherently rely on constant propagation of the accessed field for their type stability. You can write what you want to achieve like this:

function Base.setproperty!(ts::TheStruct, s::Symbol, v)
    if s === :N
        setfield!(ts, s, v) 
        arr = getfield(ts, :x) 
        resize!(arr, v)
    else
        setfield!(ts, s, v) 
    end
end

There’s probably a race condition here in multithreaded code. If I may ask, what are you trying to learn about using this?

[Ahmed_Salih]

@Sukera

Base.setproperty!(ts::TheStruct, s::Symbol, v)
    if s === :N
        setfield!(ts, s, v) 
        arr = getfield(ts, :x) 
        resize!(arr, v)
    else
        setfield!(ts, s, v) 
    end

I have a struct made as:

# Main Struct for Gradient Values
@with_kw struct ∇ᵢWᵢⱼStruct{T,ST}
    NL::Base.RefValue{Int}

    # Input for calculation
    xᵢⱼ     ::AbstractVector{SVector{ST,T}} = zeros(SVector{ST,T},NL[])
    xᵢⱼ²    ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
    dᵢⱼ     ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
    qᵢⱼ     ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
    ∇ᵢWᵢⱼ   ::AbstractVector{SVector{ST,T}} = similar(xᵢⱼ,NL[])
end

Where NL is the variable determining the length of each array. When I update NL, I wish for all other arrays to update in size as well.

To me this would be neat if it was done through updating “NL” automatically

The reason I use BaseValue.Ref etc. is explained more here:

How to properly pass structs into GPU? (MWE included)

Thank God, I was finally able to do it exactly how I wanted! And this answers my 5th question: 5. Is the way shown here the intended and “state of the art” approach for using structs in GPU programming - if not, what is? I looked at the interpolate example in the docs, and I struggle with grasping it. Short answer, no. The code below is in my opinion much better: # Packages using CUDA using Parameters using StaticArrays using Adapt using Setfield using LinearAlgebra using BenchmarkTools # Co…

So I was hoping that by updating NL I could trigger a “reaction” updating everything which is an AbstractArray in the struct to the new size

Kind regards

[Sukera]

@Ahmed_Salih

So I was hoping that by updating NL I could trigger a “reaction” updating everything which is an AbstractArray in the struct to the new size

You’ll have to implement that manually using getproperty - there is no builtin “callback” or the sort. If it’s about GPU code, I have my doubts about this being efficient though.

[Ahmed_Salih]

I think I solved it!

This works for me:

function Base.setproperty!(ts::∇ᵢWᵢⱼStruct, s::Symbol, v)
    if s === :NL
        ts.NL[] = v
        for P in propertynames(ts)
            arr = getfield(ts, P)
            if isa(arr,AbstractVector)
                resize!(arr, v)
                fill!(arr,zero(eltype(arr)))
            end
        end
    else
        @warn "You are trying to setproperty! of immutable struct. Operation cancelled"
    end
end

@benchmark setproperty!($ts,$(:NL),$(10^5))
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  177.000 μs …   6.790 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     187.900 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   218.299 μs ± 124.975 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▅█▅▃▂▃▃▁▁▁    ▁▁                                              ▁
  ███████████████████▇▇▆▆▆▆▇▆▅▅▆▆▆▅▅▅▅▆▅▆▅▅▅▅▄▅▆▅▅▄▅▄▄▄▅▅▃▃▅▄▄▄ █
  177 μs        Histogram: log(frequency) by time        641 μs <

 Memory estimate: 3.91 KiB, allocs estimate: 95.

Looks good to you?

Kind regards

[dylanxyz]

Why don’t you overload Base.resize! for your type?

I think this is more idiomatic:

function Base.resize!(object::TheStruct, N::Integer)
    object.N = N
    resize!(object.x, N)
    # do other things here
end

Then, you can just call resize! instead of manually updating N with the dot syntax.

[Ahmed_Salih]

@dylanxyz

function Base.resize!(object::TheStruct, N::Integer)
    object.N = N
    resize!(object.x, (N,))
    # do other things here
end

I think this is the actual solution I was looking for! Thank you for realizing that for me.

And thank you @Sukera for showing me how I would be able to do it the way you showed me, even though it was quite “hacky”. Performance of both of them is very similar:

julia> @CUDA.time setproperty!(ts,:NL,10^5)
  0.001449 seconds (153 CPU allocations: 7.781 KiB) (5 GPU allocations: 3.433 MiB, 3.41% memmgmt time)

julia> @CUDA.time resize!(ts,10^5)
  0.001277 seconds (144 CPU allocations: 7.016 KiB) (5 GPU allocations: 3.433 MiB, 3.99% memmgmt time)

Resize wins out slightly.

I don’t get why it allocates though on the GPU, I am using in-place operations?

The code is:

function Base.resize!(object::∇ᵢWᵢⱼStruct, N::Integer)
    object.NL[] = N
    for P in propertynames(object)
        arr = getfield(ts, P)
        if isa(arr,AbstractVector)
            resize!(arr, N)
            fill!(arr,zero(eltype(arr)))
        end
    end
end

Kind regards

[Sukera]

@Ahmed_Salih

I don’t get why it allocates though on the GPU, I am using in-place operations?

Resizing may need to allocate a new array, if the new size is too large. This includes copying the old data into the new array.

[Ahmed_Salih]

Okay, thank you, I thought I would just change the length of the container and then it would not need to allocate any new memory…

Will live with this for now and see if I can avoid it some way later

Kind regards

[Henrique_Becker]

I think that to guarantee no allocations you have to allocate a “big enough” vector and then hand out slices. Otherwise resize! may need to allocate more memory, I am not sure if resize! ever decides to do it when downsizing, so maybe you could just use resize! if you already start at the largest size possible, but I am not sure.

[Ahmed_Salih]

Okay, I think I like that idea!

So what you are in a sense proposing;

1. If I know that N is at max 120k, then I preallocate this size of array
2. In my overloaded resize operation, I instead return the “valid indices” for this step of my calculation
3. If the (new N) > (old N) then I do the allocation of memory of course

So in this way I will only need to allocate memory when I actually need it and the values above “current N” are “trash”.

I really like this approach since I also just found out that I can set the values of the valid indices using a @view to get 0 GPU allocs:

@CUDA.time fill!(@view(V_GPU_Kernel.dᵢⱼ[1:50000]),100)
0.001678 seconds (60 CPU allocations: 3.688 KiB)

# And then

V_GPU_Kernel.dᵢⱼ
100000-element CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}:
 100.0
 100.0
 100.0
 100.0
 100.0
 100.0
 100.0
 100.0
 100.0
 100.0
 100.0
   ⋮
   0.0
   0.0
   0.0
   0.0
   0.0
   0.0
   0.0
   0.0
   0.0
   0.0

[Henrique_Becker]

Yes, this is what I was proposing. If views work for your application, and keeping the maximum size allocated is not a problem, then it is a good solution.

I do not understand how GPU programming work, so just to be clear:

1. resize! often allocates more than the n you give it, so future resizes will not need to re-allocate and copy, but you only have access to indices up to n (in C++ vectors this is the difference between size and capacity).
2. resize! does this re-allocation transparently, the Vector just gets bigger, it is the same object in Julia and you do not need to adapt nothing in your side, but the underlying C pointer may be pointing to a different memory region and low-level libraries/GPU maybe can have a problem with this, I do not know.

[Ahmed_Salih]

Great!

I finished the first implementation, I added a “MaxValidIndex” field to my struct:

@with_kw struct ∇ᵢWᵢⱼStruct{T,ST}
    NL::Base.RefValue{Int}
    MaxValidIndex::Base.RefValue{Int} = deepcopy(NL)

    # Input for calculation
    xᵢⱼ     ::AbstractVector{SVector{ST,T}} = zeros(SVector{ST,T},NL[])
    xᵢⱼ²    ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
    dᵢⱼ     ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
    qᵢⱼ     ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
    ∇ᵢWᵢⱼ   ::AbstractVector{SVector{ST,T}} = similar(xᵢⱼ,NL[])
end

And the resize! function is now as follows:

function Base.resize!(object::∇ᵢWᵢⱼStruct, N::Integer)
    if N > object.NL[]
        object.NL[] = N
        for P in propertynames(object)
            arr = getfield(object, P)
            if isa(arr,AbstractVector)
                resize!(arr, N)
                fill!(arr,zero(eltype(arr)))
            end
        end
    else
        object.MaxValidIndex[] = N
        for P in propertynames(object)
            arr = getfield(object, P)
            if P !== :xᵢⱼ
                if isa(arr,AbstractVector)
                    fill!(@view(arr[1:N]),zero(eltype(arr)))
                end
            end
        end
        
    end

    return nothing
end

So basically if the new N is suffient to be held in the old array, we do not allocate a new one, but instead “clean” values up to new N (MaxValidIndex). All my functions have now been changed to respect MaxValidIndex and I know that the range from 1:MaxValidIndex will always hold the real values, everything above is “trash”.

This also did not give me any noticeable regression in performance so this is awesome. Thank you very much @Henrique_Becker for the idea, since re-allocating all the time would be devasting for my code, i.e imagine 200k iterations with 5 mb realloc each time… now limited

EDIT: Also regarding your point 1 and 2. I noticed that if I increased size of GPU array with 1, it would still allocate a whole new one. It does not seem to be “smart” in the background as on CPU

Kind regards