StaticArrays and allocations

Note that the macro of Setfield that “mutates” is @set!:

julia> using StaticArrays

julia> x = @SVector [1,1,1]
3-element SVector{3, Int64} with indices SOneTo(3):
 1
 1
 1

julia> using Setfield

julia> @set x[1] = 2
3-element SVector{3, Int64} with indices SOneTo(3):
 2
 1
 1

julia> x
3-element SVector{3, Int64} with indices SOneTo(3):
 1
 1
 1

julia> @set! x[1] = 2
3-element SVector{3, Int64} with indices SOneTo(3):
 2
 1
 1

julia> x
3-element SVector{3, Int64} with indices SOneTo(3):
 2
 1
 1

But, @paulmelis, it is probably useful to think about these immutable objects as single values most of the time. This makes most code very readable. For instance:

julia> positions = [ @SVector rand(3) for i in 1:1000 ];

julia> velocities = [ @SVector rand(3) for i in 1:1000];

julia> function update_positions!(x,v,Δt)
         for i in eachindex(x)
           x[i] = x[i] + v[i]*Δt
         end
       end
update_positions! (generic function with 1 method)

julia> using BenchmarkTools

julia> @btime update_positions!($positions,$velocities,0.1)
  1.300 μs (0 allocations: 0 bytes)

Or, even simpler:

julia> update_positions!(x,v,Δt) = @. x = x + v*Δt
update_positions! (generic function with 1 method)

julia> @btime update_positions!($positions,$velocities,0.1)
  1.198 μs (0 allocations: 0 bytes)

which is furthermore slightly faster because inbounds is granted.

2 Likes

Neither of them mutate, @set! is just a convenience macro to rename the variable.

2 Likes

Sure, but the user experience is different.

Yes. immutable-ness of structs or SVector doesn’t mean that e.g. + isn’t defined or that operations with them will allocate (in fact, I’m willing to bet that type stable code with SVectors in it will not allocate, unless you fall back to regular arrays somewhere along the line). It really depends on what you want to do with them - maybe @paulmelis could give use some more information about what they’re trying to do?

My point is we shouldn’t suggest that they mutate because the key point for this thread is that they are guaranteed to always live on the stack. Mutables aren’t. I think its better if users know what they do and have a clean mental model.

4 Likes

Agreeing with you, an example to reinforce what you are saying:

julia> x = @SVector [1,1,1];

julia> f!(x) = @set! x[1] = 2
f! (generic function with 1 method)

julia> f!(x)
3-element SVector{3, Int64} with indices SOneTo(3):
 2
 1
 1

julia> x
3-element SVector{3, Int64} with indices SOneTo(3):
 1
 1
 1

They “look” mutable in the scope of the function. But they aren’t.

1 Like

I’m still not clear for which of the above forms the values are guaranteed to live on the stack? Are there cases where even SVector can end up being allocated on the heap?

Are there cases where even SVector can end up being allocated on the heap?

Yeah I was too strong with “guarantee”, but your Point object will certainly be on the stack, or just registers. I do wonder what circumstances an immutable object would be heap allocated…

The short answer is none of them are guaranteed to live on the stack (or actually to even live anywhere). One major advantage julia’s compiler has over c/c++ is that it makes far fewer guarantees about how it will implement your code. C and c++ compilers are forced to compile the actual code you wrote more often, even when there was a different way that would have gotten the same algorithm faster. Julia, on the other hand gives itself more room to change the details as long as it can prove you won’t notice the difference. One example of this is that it will often not store StaticArrays anywhere in memory, and just use a few registers instead. This is an optimization c won’t often do because for it to be valid, the compiler has to prove that you haven’t done something ugly like take the address of an integer that happens to refer to the memory location of the array.

1 Like

But this is a good lack of guarantee! don’t we all want our SArrays to just use registers?

The concern about guarantees is heap allocations… is there example of an SArray being heap allocated?

I think there’s a use for guaranteed stack-allocated, size-known-at-compile-time arrays (possibly from literals only?), though I agree that it’s kind of niche if you get used to not requiring them at all. Certainly requires a different kind of thinking if you’re very used to doing this.

To go into more detail how this could be done would require knowing more about what you’re trying to do/what the code you want to port does, @paulmelis

This is certainly a double-edged sword. As you get more proficient with C/C++ you will know pretty well when to use stack-allocated versus heap-allocated values and what the performance trade-offs are. So it indeed requires more effort to learn and manage, but overall is pretty transparent as the language mandates a lot of implementation details and the effects of the different types of allocations are somewhat predictable in terms of performance, certainly intuitively. So it also becomes more natural to make those trade-offs yourself to get what you want.

Julia’s handling of how values get allocated might also not be too complex, but the interaction with type-stability and related automatic boxing/unboxing muddies the water a bit it seems. And this is were “to change the details as long as it can prove you won’t notice the difference” comes in, as performance can be greatly influenced by too many allocations, which can be influence by type-instabilities, or by use of SVector versus a normal array, or by mutable versus immutable, etc. So before you even get to the clever optimizations that the compiler does it seems there’s a slew of different higher-level choices to be made which are far more important for resulting performance, but which are not really part of the core Julia language. So here the mental modal of e.g. allocations happening in your code appears to be quite a bit more complex. Even though the Julia compiler will do a lot more clever things it also becomes more opaque and less predictable IMO. Case in point are my questions on the StaticArrays types as noted above, but looking at their docs once more I really think no guarantees are given, so the only option is to try them and see what comes out? Btw, all this certainly has to do with far less experience I have with Julia compared to C++.

2 Likes

But, I think the point made was that sometimes you want neither, you want the whole object to be optimized away. Is that also convenient to achieve in C/C++?

I think that’s only possible for stack-allocated objects in C++, where a compiler might optimize it that way. But heap-allocated ones are (probably) always present explicitly in memory. Plus, in C++ I don’t think I ever explicitly want “the whole object to be optimized away”. It’s simply not a choice I can really influence, so I don’t have to worry about it.

I’m kinda not understanding how that’s a counter argument :thinking: You can’t have it, so that’s good?

I may have missed a part there. You think it could be available?

Well, you made the argument that you wanted a certain optimization, which you can achieve in Julia, yet it comes at a cost of having to understand the how to achieve that. In C++ that optimization isn’t possible so wanting to achieve it doesn’t really apply. Perhaps with some black template magic you might be able to optimize objects away, but for 99.9% of the users I suspect that’s not really of interest to them (and only to those writing the templates, not the ones using them). The core C++ language (i.e. leaving out all the exotic stuff such as concepts, move semantics, funky template meta-programming, etc) is in many ways simpler than Julia due to the less sophisticated type system. So the choices you make as a developer appear to be simpler (but that’s again based on my very limited Julia experience).

Sorry, what does this refer to?

It refers to this sentence:

I missed that at first, and thought you said this could not happen in c++.

I came from Fortran, and in Fortran (at least the one I knew) you don’t have any control over these things. Everything looks as if it was mutable, but the compilers will decide that without the user knowing about what is going on (and sometimes in a compiler-flag dependent way). Coming from Fortran, where every type is declared, one initially feels unsafe for not annotating types, then one makes mistakes and get type instabilities, and finally one gets the way around it and starts using Julia as one should. Not having to annotate every type becomes liberating at the end, not mentioning the fact that functions become generic, thus more useful. And we learn where to focus our attention to get performant code, and where all that obsessive control we had in the other language was counterproductive. At the end, not because of the possibilities of each language in general, but because the easiness of use, explore, test, benchmark, my codes are faster in Julia than what I achieved in Fortran.

10 Likes