Is resize! always in-place?

General Usage
1

Following up from Implications of resize!

If I have a vector of Float64, e.g.

a = [0.1, 0.2, 0.3]

and I call

b = resize!(a, N)

for some integer N, is there any sort of guarantee that a===b? That is, is it safe to simply do

resize!(a, N)

and use a afterwards without having to worry about where it’s the old a or the resized a?

I haven’t found a counter-example where a===b would not hold, but it seems strange that it would be possible to resize an array, stay contiguous, and not have to move it to a new memory location. If so, ignoring the return value of resize! seems like a major foot-gun (it will work most of the time!), and it would probably be a good idea to put a warning in the documentation of that routine.

2

The specific situation where this is particularly relevant is Resize arrays in mutable structs. If the return value of resize! cannot be ignored, then @rdeits comment is not quite right: you wouldn’t be able to resize a vector contained in a struct that’s not mutable (even though the compiler won’t complain about it)

3

The ! at the end resize! is there to indicate that it modifies one or more of its arguments. Since the only other argument is an immutable, having a be modified is what does function does, as is documented:

help?> resize!                            
search: resize!                           
                                          
  resize!(a::Vector, n::Integer) -> Vector
                                          
  Resize a to contain n elements.

That resize! also returns a is a detail. Sometimes it’s useful to have a function that both modifies its argument and also return it. In fact:

julia> @edit resize!([1,2,3], 5)

opens an editor pointing to:

function resize!(a::Vector, nl::Integer)                  
    l = length(a)                                         
    if nl > l                                             
        _growend!(a, nl-l)                                
    elseif nl != l                                        
        if nl < 0                                         
            throw(ArgumentError("new length must be ≥ 0"))
        end                                               
        _deleteend!(a, l-nl)                              
    end                                                   
    return a                                              
end

where you can see that a is returned again.

1 Like
4

Well, that’s interesting… but how can _growend! possibly grow an array without changing its memory address?

1 Like
5

It doesn’t (well, not of a itself anyway). The underlying wrapped memory may be moved around (you can dig into src/array.c:896/jl_array_grow_at_end for when/how this is done if you’d like). You’ll also be interested in src/array.jl:681/array_resize_buffer, which handles the reallocation (but the caller has to handle moving the data from the old buffer into the new buffer). In general though arrays “oversubscribe” memory and have space to grow at the end (I believe this is done by doubling the allocated size once full, which gives amortized constant insertion time) without having to copy memory for every single growth.

Nonetheless, from julia itself this is transparent - a was the array before and a is the same array afterwards. Internally, only its data pointer points somewhere new now, the array itself hasn’t moved. All this is very internal and if you keep a pointer in Fortran/C/C++ to some internal julia data instead of the “wrapper” object (i.e., the thing that’s holding the pointer to the data), you’re playing with fire already anyway.

–

I want to stress here that this is inconsequential when writing julia code - a will still be a after resize!. You won’t lose that ===.

4 Likes
6

Thanks! If Julia has an internal data pointer that can change while the objectid of the Vector stays the same, that totally explains it! I was aware of the “oversubscribing” trick generally used in memory management, so I wasn’t surprised that in many cases, the resize will not require any actual new memory (the data pointer stays the same). If you more than double it, though, you eventually may have to move the data… but if Julia handles it transparently, everything is fine! It means that I can ignore the return value of resize! and more importantly, that I can resize Vectors that are stored in structs.

7

You could always do that. It doesn’t matter whether a struct itself is immutable or mutable, if a field is mutable you can always mutate that field (under the current semantics anyway). There has been some talk about creating an array type that isn’t resizable or that can be “frozen” to prevent modifications for a certain time, but those discussions are in their infancy.

Yes, you can ignore it. As you can see from the code posted above, it wouldn’t matter anyway - the same object (a) is returned again, it just has a new size.

Having resize! return its argument is useful when writing some chained operations, e.g.

# v is a vector of length 10
prod(resize!(v, length(v)-5))

will calculate the product of the first 5 elements of v and throwing away the last 5 at the same time. It’s personal preference whether to do the above or this instead:

resize!(v, length(v)-5)
prod(v)

The outcome is the same either way.

In general, trying to reason with pointers about julia code can often lead you astray - unless you know exactly that you’re dealing with pointers, it’s best not to try to think where they may be hiding under the hood.

1 Like