# Julia alignas: is there a way to specify the alignment of Julia objects in memory?

**URL:** <https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501>\
**Category:** General Usage\
**Tags:** question, simd, array, vector, cache\
**Created:** [March 18, 2021, 10:40pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501 "2021-03-18T22:40:59Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [March 18, 2021, 10:40pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/1 "2021-03-18T22:40:59Z")

</div>

In C++ there is `alignas` (and related stuff like `alignof`):  
[https://en.cppreference.com/w/cpp/language/alignas](https://en.cppreference.com/w/cpp/language/alignas)

`alignas` enables specifying, for example, that an array’s storage should be aligned to 64 bytes in memory, which is useful if it is known that the array will be large and that it will be operated on by SIMD instructions. It can also be useful when one wants to ensure that the data will fit in CPU cache nicely.

I think such features are missing from Julia? Is it preferable to open a thread in [Internals & Design - JuliaLang](https://discourse.julialang.org/c/dev/5), or to open an issue on Github?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 18, 2021, 11:04pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/2 "2021-03-18T23:04:42Z")

</div>

You can allocate an `Array` using [`posix_memalign`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/posix_memalign.html), for example:

```julia
function alignedvec(::Type{T}, n::Integer, alignment::Integer=sizeof(Int)) where {T}
    @static if Sys.iswindows()
        return Array{T}(undef, n)
    else
        ispow2(alignment) || throw(ArgumentError("$alignment is not a power of 2"))
        alignment ≥ sizeof(Int) || throw(ArgumentError("$alignment is not a multiple of $(sizeof(Int))"))
        isbitstype(T) || throw(ArgumentError("$T is not a bitstype"))
        p = Ref{Ptr{T}}()
        err = ccall(:posix_memalign, Cint, (Ref{Ptr{T}}, Csize_t, Csize_t), p, alignment, n*sizeof(T))
        iszero(err) || throw(OutOfMemoryError())
        return unsafe_wrap(Array, p[], n, own=true)
    end
end

```

but [`posix_memalign` is unavailable on Windows](https://stackoverflow.com/questions/33696092/whats-the-correct-replacement-for-posix-memalign-in-windows) so it has to fall back to unaligned memory in that case (Julia defaults to 16-byte alignment). If the alignment is just an optimization, however, that may not be so terrible.

It doesn’t seem crazy to add an `alignment` keyword argument to the `Array` constructor, but the difficulty of supporting Windows is (as usual) a pain point. (You can have an aligned-memory allocator on Windows, but then you can’t use `free` to deallocate it, and so you’d need lower-level hacks for garbage-collection to work.)

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [March 19, 2021, 5:22am UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/3 "2021-03-19T05:22:30Z")

</div>

[Array alignments](https://github.com/JuliaLang/julia/blob/107901d09a847eb43c0cde923861b01f969b428d/src/julia_internal.h):

```nohighlight
#define JL_SMALL_BYTE_ALIGNMENT 16
#define JL_CACHE_BYTE_ALIGNMENT 64

```

[Array size threshold](https://github.com/JuliaLang/julia/blob/c1c52930bb58605b60ae723c28f63e43b73944f6/src/options.h):

```nohighlight
// how much space we're willing to waste if an array outgrows its
// original object
#define ARRAY_INLINE_NBYTES (2048*sizeof(void*))

```

[Array allocation](https://github.com/JuliaLang/julia/blob/0926ed8834827d0bb32c9b5ae013ab74c3e238b9/src/array.c#L116-L128). I didn’t look too closely, but calls to functions like `jl_alloc_array_1d` forward to `_new_array_` eventually, which branches off `ARRAY_INLINE_NBYTES`, and calls `JL_ARRAY_ALIGN` to “align whole object” with either `JL_SMALL_BYTE_ALIGNMENT` or `JL_CACHE_BYTE_ALIGNMENT`.  
For `Float64`, note that

```julia
julia> 256 * sizeof(Float64)
2048

```

Experimentally, the threshold seems to be at `Vector{Float64}(undef, 245)` and larger are always aligned to 64 bytes.

```julia
julia> any((reinterpret(UInt, pointer(Vector{Float64}(undef, 245))) % 64) ≠ zero(UInt) for i ∈ 1:1000)
false

julia> any((reinterpret(UInt, pointer(Vector{Float64}(undef, 244))) % 64) ≠ zero(UInt) for i ∈ 1:1000)
true

```

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [March 19, 2021, 4:15pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/4 "2021-03-19T16:15:00Z")

</div>

> [@stevengj](#):
>
> It doesn’t seem crazy to add an `alignment` keyword argument to the `Array` constructor, but the difficulty of supporting Windows is (as usual) a pain point.

I guess adding something like [`std::assume_aligned`](https://en.cppreference.com/w/cpp/memory/assume_aligned) from C++20 is a more viable option, then? I guess it would be called something like `unsafe_assumealigned` in Julia. It would be used by the programmer to tell Julia (and then LLVM) what’s the alignment of a pointer obtained through `posix_memalign`.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 19, 2021, 4:30pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/5 "2021-03-19T16:30:58Z")

</div>

> [@nsajko](#):
>
> I guess adding something like [`std::assume_aligned`](https://en.cppreference.com/w/cpp/memory/assume_aligned) from C++20 is a more viable option, then?

You’d first have to make a compelling case for this providing a real benefit (i.e. much faster code is generated by LLVM).

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [March 20, 2021, 3:46pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/6 "2021-03-20T15:46:07Z")

</div>

I don’t think that’s the case, at least on recent x64 hardware.  
Aligned loads/stores are faster, but aligned move instructions aren’t (move instructions are used for loading/storing). It’s just that one crashes if unaligned.  
So the benefit of promising alignment isn’t performance. The benefit is free runtime checks (free from a performance perspective) that the memory really is aligned. You’ll be notified by a segfault if you’re wrong, rather than silently having worse performance.

EDIT:  
Some compilers, like gcc, will often generate alignment checks + some code to align if unaligned in front of loops. So `std::assume_aligned` could let it skip these checks, which would have a performance benefit.

Also:

```julia
using VectorizationBase: assume
function mydot_aligned(x,y)
    s = zero(promote_type(eltype(x),eltype(y)))
    assume((reinterpret(UInt, pointer(x)) % (64 % UInt)) == zero(UInt))
    assume((reinterpret(UInt, pointer(y)) % (64 % UInt)) == zero(UInt))
    @inbounds @simd for i in eachindex(x,y)
        s += x[i]*y[i]
    end
    s
end

```

produces this SIMD loop (`@code_native`):

```nohighlight
L176:
        vmovapd zmm4, zmmword ptr [rax + 8*rsi]
        vmovapd zmm5, zmmword ptr [rax + 8*rsi + 64]
        vmovapd zmm6, zmmword ptr [rax + 8*rsi + 128]
        vmovapd zmm7, zmmword ptr [rax + 8*rsi + 192]
        vfmadd231pd zmm0, zmm4, zmmword ptr [rcx + 8*rsi] # zmm0 = (zmm4 * mem) + zmm0
        vfmadd231pd zmm1, zmm5, zmmword ptr [rcx + 8*rsi + 64] # zmm1 = (zmm5 * mem) + zmm1
        vfmadd231pd zmm2, zmm6, zmmword ptr [rcx + 8*rsi + 128] # zmm2 = (zmm6 * mem) + zmm2
        vfmadd231pd zmm3, zmm7, zmmword ptr [rcx + 8*rsi + 192] # zmm3 = (zmm7 * mem) + zmm3
        add rsi, 32
        cmp rdx, rsi
        jne L176

```

Notice the `vmovapd`s instead of `vmovupd`s.  
So this does work to tell LLVM about alignment.

EDIT:  
Maybe I’m wrong:

```julia
julia> x = rand(256);

julia> y = rand(256);

julia> @btime mydot($x,$y)
  10.032 ns (0 allocations: 0 bytes)
67.11501240811893

julia> @btime mydot_aligned($x,$y)
  8.534 ns (0 allocations: 0 bytes)
67.11501240811893

julia> @btime mydot($x,$y)
  10.031 ns (0 allocations: 0 bytes)
67.11501240811893

julia> @btime mydot_aligned($x,$y)
  8.530 ns (0 allocations: 0 bytes)
67.11501240811893

```

`mydot` is the same, except I commented out the `assume`s.  
EDIT:  
restarted Julia:

```julia
julia> x = rand(256);

julia> y = rand(256);

julia> @btime mydot($x,$y)
  11.590 ns (0 allocations: 0 bytes)
63.81066585474556

julia> @btime mydot_aligned($x,$y)
  11.885 ns (0 allocations: 0 bytes)
63.81066585474556

julia> @btime mydot($x,$y)
  11.589 ns (0 allocations: 0 bytes)
63.81066585474556

julia> @btime mydot_aligned($x,$y)
  11.960 ns (0 allocations: 0 bytes)
63.81066585474556

```

Was probably just noise. Sometimes functions are just randomly faster or slower for no discernible (by me) reason in a manner that is consistent within a Julia session, but not between Julia sessions/recompilations.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [March 20, 2021, 9:22pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/7 "2021-03-20T21:22:21Z")

</div>

> [@Elrod](#):
>
> I don’t think that’s the case, at least on recent x64 hardware.

I’m currently doing some experiments, with C++20 and Clang 11 on a laptop with a recent Intel CPU, and it seems that using `alignas` and `assume_aligned` can give significant performance improvements. Basically I’m giving two huge arrays with parameterized alignment to some toy functions to process (where the result goes in one of the arrays), and for _ **some** _ of those simple functions the benefit of an alignment to 256 byte boundaries gives up to 25% throughput improvement compared to default alignment.

To be more specific, each toy function takes two bytes as input and returns a single byte; and then I apply them to the arrays.

Will post results and describe the experiments in detail after trying out some other stuff like Polly and OpenMP and making visualizations.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [March 20, 2021, 9:26pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/8 "2021-03-20T21:26:07Z")

</div>

> [@Elrod](#):
>
> `using VectorizationBase: assume`

Wait what? I can’t find `VectorizationBase` on JuliaObserver, and it’s not part of Julia proper? Is it some proof of concept implementation made by yourself?

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [March 20, 2021, 9:26pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/9 "2021-03-20T21:26:57Z")

</div>

What about `alignas` without `assume_aligned`?

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [March 20, 2021, 9:28pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/10 "2021-03-20T21:28:19Z")

</div>

Use [JuliaHub](https://juliahub.com/ui/Packages/VectorizationBase/Dto5m/0.19.12), not JuliaObserver.  
(Note, it says tests are failing because CI’s been temporarily disabled by GitHub staff)

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [March 20, 2021, 9:30pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/11 "2021-03-20T21:30:35Z")

</div>

Well, `assume_aligned` is only useful in my experiments when the compiler “forgets” about the alignment it already promised as part of the `alignas`. They’re basically complementary tools as far as I understand, but `assume_aligned` shouldn’t really make much of a difference for the effects of `alignas` conceptually.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [March 20, 2021, 9:37pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/12 "2021-03-20T21:37:37Z")

</div>

I didn’t/don’t think the compiler knowing about alignment helps, except when it lets the compiler skip alignment checks. That for unaligned loads/stores that cross vector-width boundaries, `vmovupd` will have half the throughput as when it’s aligned while `vmovapd` will segfault.  
But that if loads are aligned / don’t cross such a boundary, `vmovapd` and `vmovupd` are equally fast (i.e., twice as fast as the unaligned `vmovupd`).

Which would mean that big arrays in Julia – being aligned to 64 bytes automatically – means that you should be able to get full performance without needing to tell the compiler about it.

But they’re not aligned to 256 bytes. Out of curiosity, why do you need 256 bytes?  
64 byte would be aligned with x64 cacheline sizes as well as the largest SIMD registers on x64. 4096 with pages. Hadn’t heard of 256 bytes before, so I’m curious about the reason.

An exception would be `vmovnt`, non-temporal stores, which require alignment. But a compiler isn’t likely to use those automatically AFAIK.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [March 20, 2021, 9:41pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/13 "2021-03-20T21:41:59Z")

</div>

Oh, I don’t need 256 bytes, I’m just playing around. You’re probably correct about 256 byte alignment being unnecessary, I just chose it because with only two arrays in each experimental program, giving them too much alignment is not an issue.

> [@Elrod](#):
>
> I didn’t/don’t think the compiler knowing about alignment helps, except when it lets the compiler skip alignment checks.

I’m just making uneducated guesses, but it could be that the compiler decides not to vectorize, or to use suboptimal vectorization, if it doesn’t know about the alignment. Another guess is that the eliminated branches help with code fitting in L1 cache or something like that.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [March 20, 2021, 9:50pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/14 "2021-03-20T21:50:33Z")

</div>

Hey, I don’t really know anything about that llvmcall stuff, so I have to ask: it seems that  
`assume` is not actually used anywhere in `VectorizationBase`, so are you sure it’s actually functional?

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [March 20, 2021, 9:53pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/15 "2021-03-20T21:53:04Z")

</div>

Yes.  
From my example:

```julia
L176:
        vmovapd zmm4, zmmword ptr [rax + 8*rsi]
        vmovapd zmm5, zmmword ptr [rax + 8*rsi + 64]
        vmovapd zmm6, zmmword ptr [rax + 8*rsi + 128]
        vmovapd zmm7, zmmword ptr [rax + 8*rsi + 192]

```

Notice the [vmovapd](https://www.felixcloutier.com/x86/movapd)? These are aligned loads.

VectorizationBase mostly just defines functions for other libraries to use. ThreadingUtilities.jl and LoopVectorization.jl both use `VectorizationBase.assume`.

(EDIT: But you should always double check when using `assume`.)

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [March 20, 2021, 10:25pm UTC](https://discourse.julialang.org/t/julia-alignas-is-there-a-way-to-specify-the-alignment-of-julia-objects-in-memory/57501/16 "2021-03-20T22:25:25Z")

</div>

`valloc` in SIMD.jl allows you to specify an alignment.

```julia
using SIMD
for i ∈ 1:10
   a = valloc(UInt8, 1024, 8)
   b = valloc(UInt8, 2048, 8)
   Int(pointer(a)) % 2048 == 0 || println("a not aligned to 2048")
   Int(pointer(b)) % 2048 == 0 || println("b not aligned to 2048")
end

```