FixedSizeVectors with cache-aligned memory?

General Usage
,
1

FixedSizeVectors offer a first, more performant and cleaner approach to working with (generally, numeric) vectors where, once formed, the size of the vector does not change.
AlignedAllocs provides cache-aligned memory as a Vector{T} where T is a given bitstype:

using FixedSizeArrays
using AlignedAllocs: memalign_clear, alignment  
   # memalign_clear zeros the memory block
   # also exports `memalign` for a memory block of type T that is uninitialized.
   # alignment(xs::AbstractVector) provides the byte alignment of the initial element of xs
   #                                                 and so the byte alignment of the vector

element_type = Float32
element_count = 8

cache_aligned_mem = memalign_clear(element_type, element_count)
fixed_length_mem     = FixedSizeVectorDefault{element_type}(undef, element_count)

alignment(cache_aligned_mem) >= 64
alignment(fixed_length_mem)     >= 32  # for vectors with fewer than 2^9 elements
                                                                # on my machine

Working with cache-line-size aligned vectors is more perfomant generally and prevents the possible time hijacking of cache-line crossing data where it need not cross cache lines.
Most current pcs have a cache-line-size of 64 bytes (there are exceptions).
When using SIMD operations, aligning to 256 or 512 bytes (processor dependant) is very important to the performance of SIMD driven algorithms.

cache_aligned_mem = memalign_clear(element_type, element_count, 256)
alignment(cache_aligned_mem) >= 256

It would be great to obtain a FixedLengthVector backed by cache-aligned memory.

2 Likes
2

It has nothing to do with the implementation of AlignedAllocs, the FixedSizeArray type just doesn’t (yet) support directly wrapping an input DenseVector with a possibly different size. It’s just some uninitialized instantiation and collecting input AbstractArrays into new parent DenseVectors (currently defaulting to Memory), which is a bit strange because the latter seems to be what FixedSizeArrayDefault is intended for.

julia> methods(FixedSizeArray.body.body.body)
# 4 methods for type constructor:
 [1] (::Type{FSA})(::UndefInitializer, size::Tuple{Vararg{Integer}}) where {T, FSA<:(FixedSizeArray{T, N, Mem} where {N, Mem<:DenseVector{T}})}
     @ C:\Users\benm1\.julia\packages\FixedSizeArrays\22QFl\src\FixedSizeArray.jl:138
 [2] (::Type{FSA})(::UndefInitializer, size::Integer...) where {T, FSA<:(FixedSizeArray{T, N, Mem} where {N, Mem<:DenseVector{T}})}
     @ C:\Users\benm1\.julia\packages\FixedSizeArrays\22QFl\src\FixedSizeArray.jl:141
 [3] (::Type{FSA})(src::AbstractArray) where {N, FSA<:(FixedSizeArray{var"#s3", N, Mem} where {var"#s3", Mem<:DenseVector{var"#s3"}})}
     @ C:\Users\benm1\.julia\packages\FixedSizeArrays\22QFl\src\FixedSizeArray.jl:336
 [4] (::Type{FSA})(src::AbstractArray) where FSA<:FixedSizeArray
     @ C:\Users\benm1\.julia\packages\FixedSizeArrays\22QFl\src\FixedSizeArray.jl:343

We can verify with a minimal example type:

julia> begin
       # might be an incorrect subtype, but it'll serve the example
       mutable struct OneElement{T} <: DenseVector{T} value::T end
       Base.size(::OneElement) = (1,)
       Base.getindex(A::OneElement, i::Int) = if i == 1 A.value else throw(BoundsError(A, i)) end
       Base.setindex!(A::OneElement, v, i::Int) = if i == 1 A.value = v else throw(BoundsError(A, i)) end
       using FixedSizeArrays
       end

julia> FixedSizeArray(OneElement(2f0)) # input collected into parent Memory
1-element FixedSizeArray{Float32, 1, Memory{Float32}}:
 2.0

julia> FixedSizeVector(OneElement(2f0)) # of course aliases also do this
1-element FixedSizeArray{Float32, 1, Memory{Float32}}:
 2.0

There is only a private inner constructor(?) that can do this, but it’s not ideal because there isn’t an option to compute the size directly from the DenseVector and broadcasting fails at a with_stripped_type_parameters_unchecked call that is only implemented for Vector and GenericMemory.

julia> FixedSizeArrays.new_fixed_size_array(OneElement(2f0), (1,)) # no idea why the alias is printed
1-element FixedSizeVector{Float32, OneElement{Float32}}:
 2.0

julia> 1 .+ FixedSizeArrays.new_fixed_size_array(OneElement(2f0), (1,1))
ERROR: MethodError: no method matching with_stripped_type_parameters_unchecked(::FixedSizeArrays.TypeParametersElementType, ::Type{OneElement{Float32}})
3

One relevant feature request to Julia:

Assuming wrap were to get publicized without any changes to the interfaces, FixedSizeArrays.jl could add its method to wrap, and what you want would be possible for Julia v1.11 and up. On the other hand, if the wrap interface changes before wrap gets publicized, I suppose FixedSizeArrays.jl could provide its own function to enable wrapping functionality.

4

It’s not ideally documented, but you can do e.g. vv=[3]; FixedSizeArrays.new_fixed_size_array(vv.ref.mem, (3,)). As my example demonstrates, be careful, for some weird reason FixedSizeArrays decided not to check sizes.

(so you posix_memalign or analogue to allocate your stuff, then unsafe_wrap(Memory...) to get a Memory instance, and then FixedSizeArrays.new_fixed_size_array(mem, size)).