Julian way of expressing a buffer swap

Coming from C++, I’m used to writing something like

class workspace {
    public:
    int* frontBuffer;
    int* backBuffer;

   workspace() { ... allocate front and back buffers here ...}

   void swap_buffers() {
     int* tempPtr = frontBuffer;
     frontBuffer = backBuffer;
     backBuffer = tempPtr;    
   }
};

With the idea that operations on the buffers can read from frontBuffer, write to backBuffer and then swap the buffers before returning, leaving their result in the frontBuffer for the next operation.

What is the Julian way of performing this operation? Should I use a mutable struct? My suspicion is I should not, and instead have something like

struct Workspace
    frontBuffer::Array{Int64}
    backBuffer::Array{Int64}
end

function Workspace(s::Int64)
    return Workspace(zeros(s), zeros(s))
end

function swapbuffers(w::Workspace)
    return Workspace(w.backBuffer, w.frontBuffer)
end

This requires that each operation return the updated workspace and means that if I do

julia> w = Workspace(5)
Workspace([0, 0, 0, 0, 0], [0, 0, 0, 0, 0])

julia> w.frontBuffer[1] = 1
1

julia> w
Workspace([1, 0, 0, 0, 0], [0, 0, 0, 0, 0])

julia> r = swapbuffers(w)
Workspace([0, 0, 0, 0, 0], [1, 0, 0, 0, 0])

julia> w
Workspace([1, 0, 0, 0, 0], [0, 0, 0, 0, 0])

Of course, while I’ve allocated a new Workspace there’s been no additional allocations of the underlying arrays, as

julia> w.backBuffer === r.frontBuffer
true

Again, coming from C++, I’m used to passing a reference to my workspace as an argument to an operation, and so there would be no new allocation of a Workspace at each swap, and the buffer swap would be visible to anyone holding a reference to the underlying workspace (i.e., w.backBuffer === r.backBuffer would be true).

I understand that there’s tradeoffs between these approaches, and I’m not asking about which is better, or how to make my Julia code like my C++ code. I admit I’m a little suspicious of allocating a new Workspace at each swap when I expect to do millions of these as part of a big computation, but I’m open to the idea that this is basically irrelevant due to good optimization and garbage collection. I’m mostly just wondering whether the Julia approach I’ve described above is the “right way” to do this kind of thing in Julia, or if there’s some other idiom I should be looking at.

Typically Julians avoid allocation when there’s no real reason to allocate. So unless you need separate Workspace objects for some reason, you’d probably use a mutable struct and just do

w.frontBuffer,w.backBuffer = w.backBuffer,w.frontBuffer

But I’d like to see what others say… I’m not convinced I’m right on that.

1 Like

Little aside, note that you haven’t specified the dimension of those arrays here. Array{Float64} is an abstract type, you’re probably looking for Array{Float64, 2} (or equivalently, Matrix{Float64}).

3 Likes

You’d be correct - immutable structs basically don’t allocate and can live mostly on the stack. Having an immutable struct and “recreating” it over and over isn’t a problem in julia. You can convince yourself of that by benchmarking some non-trivial function with e.g. BenchmarkTools.

That being said, using a mutable struct and modifying it is fine as well. It really depends on your performance requirements for how you want to use this struct.

4 Likes

If the mutable struct doesn’t escape, it will generally be stack allocated as well.
This is not true of Arrays, however.

3 Likes
mutable struct Workspace
    frontBuffer::Vector{Int64}
    backBuffer::Vector{Int64}
end

function Workspace(s::Int64)
    return Workspace(zeros(Int64, s), zeros(Int64, s))
end

function swapbuffers!(w::Workspace)
    w.backBuffer, w.frontBuffer = w.frontBuffer, w.backBuffer
    return w
end

Now:

julia> w = Workspace(5)
Workspace([0, 0, 0, 0, 0], [0, 0, 0, 0, 0])

julia> w.frontBuffer[1] = 1
1

julia> w
Workspace([1, 0, 0, 0, 0], [0, 0, 0, 0, 0])

julia> r = swapbuffers!(w)
Workspace([0, 0, 0, 0, 0], [1, 0, 0, 0, 0])

julia> w
Workspace([0, 0, 0, 0, 0], [1, 0, 0, 0, 0])
1 Like

Okay, it makes sense that since these are just pushed/popped with the stack frames, any memory copies come from register spilling at the function boundaries. Do the variables of the Workspace struct live in registers if theres only a few of them? I guess I’m not clear on the relative performance of a stack-allocated struct vs a heap-allocated struct. My intuition is that when Workspace fits in registers it’s basically free on the stack; once it the struct gets large and exceeds available registers, I’m better having it allocated on the heap and just having a reference to it being passed on the stack. Does this sound right, or am I misunderstanding how this works?

I’m very new to Julia; can you point me to something clarifying what you mean by “escape” in this context?

1 Like

Doesn’t get returned, doesn’t get passed as an argument to a function that is not inlined.
Doesn’t get stored, either.

2 Likes

Obviously this appeals to my C++ brain, but just wanted to confirm it’s not an anti-pattern for Julia code.

1 Like

His example uses Array{Int64, 1}.

Anyway, you definitely should specify the dimensionality, @mdtisdall, or have it as a type parameter.

In my real code I do specify that as Array{Float64, 3}. My efforts to pare things down for the MWE were a bit excessive; I can put the dimension back in the example if helpful.

It’s not about being helpful (the intent is clear after all), it’s that abstractly typed fields can lead to type instability and subsequent loss of performance. It’s a common beginner mistake, so I thought I’d mention it.

3 Likes

IMO, it’s fine.

1 Like

That’s not too easy to answer - speaking purely semantically, it doesn’t matter. Generally though, if a struct is immutable and consists of only isbits, yes. For structs containing some mutable container/struct, there may be a reference to that data involved. This of course assumes that the called function is not inlined - if it is, nothing of the sort of passing around will be necessary. It should be noted that julia is quite aggressive when it comes to inlining. For more information on julias’ internal calling conventions, you can check out the developer documentation on calling conventions.

I should note though that trying to think in terms of registers when looking at julia source code is most likely not helpful at all (and will probably give you slightly wrong intuition) about when julia is fast and when it’s slow. It’s more helpful knowing when julia will SIMD and when it will not, when it will allocate and when it won’t, as well as how your data is laid out in memory to take more advantage of cache coherence.

2 Likes

Thanks! My intuition here is built from compiler courses 20+ years ago, and I know that, in practice benchmarking is better than guessing anyway. However, I’ll look at those docs to help get a better feel for how Julia’s stack works.

I feel like I’ve got a good sense of cache coherence and SIMD-friendly layout from writing C++/BLAS code. It’s the “when will Julia allocate” and type inference stuff that I’m still getting my head around.

One reason to prefer a normal struct is that mutable structs make multithreaded code harder to write.

3 Likes

Yes, I was thinking of the ”state as immutable struct” version as enforcing a logic similar to a state monad in Haskell, which certainly makes thread safety easier to enforce.