# Automatic resizing of struct internal array when value changes?

Hello!

Suppose I have a struct as such:

``````mutable struct TheStruct
N::Int
x::AbstractVector{Float32}
end
``````

And at different times “N” might change, for example from 5 to 7. When this happens I want to `resize!(x,N)` using the updated value of N = 7.

Is there a way to do this automatically, such there is some kind of “listener”?

Kind regards

Not in general, no. You can overload `setproperty!` to also `resize` the array when `N` changes, but that won’t prevent a determined user from bypassing your invariant using `setfield!`.

That sounds good to me, I am doing this mostly to learn and give it a shot! I think it could be useful for me.

Would you happen to know how to write such an overload of `setproperty!`?

Kind regards

Both `getproperty` and `setproperty!` are kind of fiddly to get right, since they inherently rely on constant propagation of the accessed field for their type stability. You can write what you want to achieve like this:

``````function Base.setproperty!(ts::TheStruct, s::Symbol, v)
if s === :N
setfield!(ts, s, v)
arr = getfield(ts, :x)
resize!(arr, v)
else
setfield!(ts, s, v)
end
end
``````

There’s probably a race condition here in multithreaded code. If I may ask, what are you trying to learn about using this?

1 Like

I have a struct made as:

``````# Main Struct for Gradient Values
@with_kw struct ∇ᵢWᵢⱼStruct{T,ST}
NL::Base.RefValue{Int}

# Input for calculation
xᵢⱼ     ::AbstractVector{SVector{ST,T}} = zeros(SVector{ST,T},NL[])
xᵢⱼ²    ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
dᵢⱼ     ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
qᵢⱼ     ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
∇ᵢWᵢⱼ   ::AbstractVector{SVector{ST,T}} = similar(xᵢⱼ,NL[])
end
``````

Where NL is the variable determining the length of each array. When I update NL, I wish for all other arrays to update in size as well.

To me this would be neat if it was done through updating “NL” automatically

The reason I use BaseValue.Ref etc. is explained more here:

So I was hoping that by updating NL I could trigger a “reaction” updating everything which is an AbstractArray in the struct to the new size

Kind regards

You’ll have to implement that manually using `getproperty` - there is no builtin “callback” or the sort. If it’s about GPU code, I have my doubts about this being efficient though.

I think I solved it!

This works for me:

``````function Base.setproperty!(ts::∇ᵢWᵢⱼStruct, s::Symbol, v)
if s === :NL
ts.NL[] = v
for P in propertynames(ts)
arr = getfield(ts, P)
if isa(arr,AbstractVector)
resize!(arr, v)
fill!(arr,zero(eltype(arr)))
end
end
else
@warn "You are trying to setproperty! of immutable struct. Operation cancelled"
end
end
``````
``````@benchmark setproperty!(\$ts,\$(:NL),\$(10^5))
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
Range (min … max):  177.000 μs …   6.790 ms  ┊ GC (min … max): 0.00% … 0.00%
Time  (median):     187.900 μs               ┊ GC (median):    0.00%
Time  (mean ± σ):   218.299 μs ± 124.975 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

▅█▅▃▂▃▃▁▁▁    ▁▁                                              ▁
███████████████████▇▇▆▆▆▆▇▆▅▅▆▆▆▅▅▅▅▆▅▆▅▅▅▅▄▅▆▅▅▄▅▄▄▄▅▅▃▃▅▄▄▄ █
177 μs        Histogram: log(frequency) by time        641 μs <

Memory estimate: 3.91 KiB, allocs estimate: 95.
``````

Looks good to you?

Kind regards

Why don’t you overload `Base.resize!` for your type?

I think this is more idiomatic:

``````function Base.resize!(object::TheStruct, N::Integer)
object.N = N
resize!(object.x, N)
# do other things here
end
``````

Then, you can just call `resize!` instead of manually updating `N` with the dot syntax.

4 Likes

I think this is the actual solution I was looking for! Thank you for realizing that for me.

And thank you @Sukera for showing me how I would be able to do it the way you showed me, even though it was quite “hacky”. Performance of both of them is very similar:

``````julia> @CUDA.time setproperty!(ts,:NL,10^5)
0.001449 seconds (153 CPU allocations: 7.781 KiB) (5 GPU allocations: 3.433 MiB, 3.41% memmgmt time)

julia> @CUDA.time resize!(ts,10^5)
0.001277 seconds (144 CPU allocations: 7.016 KiB) (5 GPU allocations: 3.433 MiB, 3.99% memmgmt time)
``````

Resize wins out slightly.

I don’t get why it allocates though on the GPU, I am using in-place operations?

The code is:

``````function Base.resize!(object::∇ᵢWᵢⱼStruct, N::Integer)
object.NL[] = N
for P in propertynames(object)
arr = getfield(ts, P)
if isa(arr,AbstractVector)
resize!(arr, N)
fill!(arr,zero(eltype(arr)))
end
end
end
``````

Kind regards

1 Like

Resizing may need to allocate a new array, if the new size is too large. This includes copying the old data into the new array.

2 Likes

Okay, thank you, I thought I would just change the length of the container and then it would not need to allocate any new memory…

Will live with this for now and see if I can avoid it some way later

Kind regards

1 Like

I think that to guarantee no allocations you have to allocate a “big enough” vector and then hand out slices. Otherwise `resize!` may need to allocate more memory, I am not sure if `resize!` ever decides to do it when downsizing, so maybe you could just use `resize!` if you already start at the largest size possible, but I am not sure.

Okay, I think I like that idea!

So what you are in a sense proposing;

1. If I know that N is at max 120k, then I preallocate this size of array
2. In my overloaded resize operation, I instead return the “valid indices” for this step of my calculation
3. If the (new N) > (old N) then I do the allocation of memory of course

So in this way I will only need to allocate memory when I actually need it and the values above “current N” are “trash”.

I really like this approach since I also just found out that I can set the values of the valid indices using a @view to get 0 GPU allocs:

``````@CUDA.time fill!(@view(V_GPU_Kernel.dᵢⱼ[1:50000]),100)
0.001678 seconds (60 CPU allocations: 3.688 KiB)

# And then

V_GPU_Kernel.dᵢⱼ
100000-element CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}:
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
⋮
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
``````
1 Like

Yes, this is what I was proposing. If views work for your application, and keeping the maximum size allocated is not a problem, then it is a good solution.

I do not understand how GPU programming work, so just to be clear:

1. `resize!` often allocates more than the `n` you give it, so future resizes will not need to re-allocate and copy, but you only have access to indices up to `n` (in C++ vectors this is the difference between `size` and `capacity`).
2. `resize!` does this re-allocation transparently, the `Vector` just gets bigger, it is the same object in Julia and you do not need to adapt nothing in your side, but the underlying C pointer may be pointing to a different memory region and low-level libraries/GPU maybe can have a problem with this, I do not know.
1 Like

Great!

I finished the first implementation, I added a “MaxValidIndex” field to my struct:

``````@with_kw struct ∇ᵢWᵢⱼStruct{T,ST}
NL::Base.RefValue{Int}
MaxValidIndex::Base.RefValue{Int} = deepcopy(NL)

# Input for calculation
xᵢⱼ     ::AbstractVector{SVector{ST,T}} = zeros(SVector{ST,T},NL[])
xᵢⱼ²    ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
dᵢⱼ     ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
qᵢⱼ     ::AbstractVector{T}             = similar(xᵢⱼ,T,NL[])
∇ᵢWᵢⱼ   ::AbstractVector{SVector{ST,T}} = similar(xᵢⱼ,NL[])
end
``````

And the resize! function is now as follows:

``````function Base.resize!(object::∇ᵢWᵢⱼStruct, N::Integer)
if N > object.NL[]
object.NL[] = N
for P in propertynames(object)
arr = getfield(object, P)
if isa(arr,AbstractVector)
resize!(arr, N)
fill!(arr,zero(eltype(arr)))
end
end
else
object.MaxValidIndex[] = N
for P in propertynames(object)
arr = getfield(object, P)
if P !== :xᵢⱼ
if isa(arr,AbstractVector)
fill!(@view(arr[1:N]),zero(eltype(arr)))
end
end
end

end

return nothing
end
``````

So basically if the new N is suffient to be held in the old array, we do not allocate a new one, but instead “clean” values up to new N (MaxValidIndex). All my functions have now been changed to respect MaxValidIndex and I know that the range from 1:MaxValidIndex will always hold the real values, everything above is “trash”.

This also did not give me any noticeable regression in performance so this is awesome. Thank you very much @Henrique_Becker for the idea, since re-allocating all the time would be devasting for my code, i.e imagine 200k iterations with 5 mb realloc each time… now limited

EDIT: Also regarding your point 1 and 2. I noticed that if I increased size of GPU array with 1, it would still allocate a whole new one. It does not seem to be “smart” in the background as on CPU

Kind regards

2 Likes