# Aligned Julia Arrays?

**URL:** <https://discourse.julialang.org/t/aligned-julia-arrays/10993>\
**Category:** Performance\
**Created:** [May 18, 2018, 2:40pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993 "2018-05-18T14:40:31Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [May 18, 2018, 2:40pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/1 "2018-05-18T14:40:31Z")

</div>

Hi,  
Is it possible to control the data alignment of Julia Arrays (for example to enable direct use of  
the `vloada` function of SIMD.jl package) ?

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [May 18, 2018, 3:25pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/2 "2018-05-18T15:25:27Z")

</div>

What do you mean?  
Julia arrays of `isbits` types are stored in column major format.  
If you want to change this, you could define your own array type and specify a different getindex function.

---

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [May 18, 2018, 3:36pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/3 "2018-05-18T15:36:43Z")

</div>

I was unclear.  
I wonder if it is possible to control the alignment of the memory allocated for Julia Arrays to enforce the first array element address to be a multiple of N=(32) bytes for AVX2.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [May 18, 2018, 3:48pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/4 "2018-05-18T15:48:14Z")

</div>

This is enforced by a (Julia) compile-time constant for `Array` instantiation right now — the first element is aligned to 16 bytes. So, no, it’s not terribly easy to control that.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [May 18, 2018, 4:33pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/5 "2018-05-18T16:33:46Z")

</div>

I would not worry about this too much. If you only care about vectorized code, most aligned instruction should just work fine on unaligned data and simply become slow.

As far as I understood, large arrays are always `JL_CACHE_BYTE_ALIGNMENT = 64` byte aligned, if they are longer than `ARRAY_INLINE_NBYTES = (2048*sizeof(void*))` = 16 KB. So if you are adressing into a large matrix, then you always control the alignment.

Short testing (on linux 64 bit, both 0.62 and 0.7) got me better alignment in practice most of the time for even smaller arrays:

```julia
julia> for i=1:1000 AA=zeros(UInt8,1952); assert(0x40== pointer(AA)-pointer_from_objref(AA)); assert(0==reinterpret(UInt64,pointer(AA)) & UInt64(63)) end
julia> for i=1:1000 AA=zeros(UInt8,1951); assert(0x40== pointer(AA)-pointer_from_objref(AA)); assert(0==reinterpret(UInt64,pointer(AA)) & UInt64(63)) end
ERROR: AssertionError:

```

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 18, 2018, 5:10pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/6 "2018-05-18T17:10:59Z")

</div>

> [@foobar\_lv2](#):
>
> As far as I understood, large arrays are always `JL_CACHE_BYTE_ALIGNMENT = 64` byte aligned,

Do you know if that is set to 128 for the POWER architecture? (if not, it should be)

---

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [May 18, 2018, 5:30pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/7 "2018-05-18T17:30:28Z")

</div>

OK, thank for your answers. So it is 16 bytes (minimum) or 64 bytes for large arrays.  
I don’t really understand this:

> [@foobar\_lv2](#):
>
> I would not worry about this too much. If you only care about vectorized code, most aligned instruction should just work fine on unaligned data and simply become slow.

I believe that an aligned load instruction will crash on unaligned data. In addition, _becoming slow_ is quite problematic if you engage a vectorization process…

Anyway thank you for indicating the JL\_CACHE\_BYTE\_ALIGNMENT and ARRAY\_INLINE\_NBYTES constants. It will help me to find how the allocation is done.

I guess that this choice for allocation has been carefully made. I clearly do not know what would imply to allow more control on this choice. If it could be done I believe that it could be useful for vectorization via package like SIMD.jl.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [May 18, 2018, 5:31pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/8 "2018-05-18T17:31:33Z")

</div>

I think not, I found the original question interesting and looked it up in the source code [here](https://github.com/JuliaLang/julia/blob/96b28d64bdd3ae037378028d4cd60e5a4c397256/src/options.h#L17).

However, the allocator will often produce better alignment than requested, just as shown in the experiment. I haven’t read the code of the impressively fast pool allocator, but if the better alignments for shorter arrays are in fact reliable (and not just true in practice) one might document this behavior.

---

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [May 18, 2018, 5:33pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/9 "2018-05-18T17:33:58Z")

</div>

IMHO _often_ is problematic because you can’t use aligned access instructions in your code.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [May 18, 2018, 5:41pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/10 "2018-05-18T17:41:04Z")

</div>

> [@LaurentPlagne](#):
>
> It will help me to find how the allocation is done.

It’s done [here](https://github.com/JuliaLang/julia/blob/4be51b7bb979e850484f9143e3b9fe84a3b5d1eb/src/array.c#L50).

> [@LaurentPlagne](#):
>
> I guess that this choice for allocation has been carefully made.

No, I think it’s black magic, and we could get better alignment for free: I think that the pool allocator doesn’t like to mix tiny and medium objects anyway, and medium-sized objects appear to get good alignment today. Maybe it’s just a documentation issue? But I have not yet read the code for the pool allocator.

> [@LaurentPlagne](#):
>
> believe that an aligned load instruction will crash on unaligned data.

What instructions do you plan to use? As far as I understood, some aligned vector load instructions check alignment and crash, but other vector loads are just as fast for aligned data (but become slow if alignment fails). For this one, you should really look at the processor manual / google for explanations and cycle counts. I don’t know the relevant instructions by heart, normally not an assembly programmer.

---

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [May 18, 2018, 5:48pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/11 "2018-05-18T17:48:50Z")

</div>

Thank you for the link.  
Ok, now I think I understand. You say that I can use the safe `vload` instruction of the SIMD.jl package instead of the unsafe `vloada` without significant performance penalty. It probably depends on the target architecture. It was not the case in the past but I must experiment on current architecture.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [May 18, 2018, 5:58pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/12 "2018-05-18T17:58:13Z")

</div>

The other possibility is to check the alignment in the beginning and branch to a different implementation if it is misaligned. Or use scalar instructions for the first few array elements until you get to an element with the desired alignment, then switch to SIMD.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 18, 2018, 6:04pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/13 "2018-05-18T18:04:26Z")

</div>

For shorter things, that can slow things down enough to be significant.  
I’m hoping that at some point, small strings can be better aligned (they are always _misaligned_ currently,  
because of the 4/8 byte length, that is at the start of each allocated 16 byte chunk.  
It might make sense to allocate small strings out of a pool that always allocates 16-byte chunks at an offset of 8.

---

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [May 18, 2018, 6:16pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/14 "2018-05-18T18:16:22Z")

</div>

Yes, but it is more work and an additional test that can cost a few cycles 😄  
The constants defined in src/julia\_internal.h should allow me to study the merit of larger alignment constraints.  
If am able to show some cases where larger alignments provide significative performance improvements, I guess that it may be a sound base to discuss the opportunity for an evolution of the Julia’s allocation alignment policy.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [May 18, 2018, 6:33pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/15 "2018-05-18T18:33:54Z")

</div>

> [@LaurentPlagne](#):
>
> You say that I can use the safe `vload` instruction of the SIMD.jl package instead of the unsafe `vloada` without significant performance penalty. It probably depends on the target architecture. It was not the case in the past but I must experiment on current architecture.

Don’t trust stuff I half-remember, experiment, google and consult the processor manual. I really am no uarch expert.

But yes, that’s what I was saying.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 18, 2018, 8:38pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/16 "2018-05-18T20:38:12Z")

</div>

One important thing to note: the older instructions (SSE, maybe SSE2?) require 16-byte alignment in many cases.

---

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [May 19, 2018, 8:31am UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/17 "2018-05-19T08:31:51Z")

</div>

OK. I only consider to increase the alignment constraint to 32 or 64 bytes for arrays. The problem of doing it globally via the internal definition of Julia is that it will waste some memory space for small arrays and may reduce global performance of applications. But it is not a problem for local experiments.

I have implemented a C++ ND-array library called Legolas++ which allow to control the underlying data layout with type parameters ([slides](https://www.researchgate.net/publication/317845331_Portable_Vectorization_and_Parallelization_of_C_Multi-dimensional_Array_Computations_Slides)). The idea was to allow for automatic vectorization of arbitrary algorithms applied to a collection of equally sized problems. I think that it can be efficiently written in Julia but the control of the memory alignment would greatly simplify this work.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 19, 2018, 8:46am UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/18 "2018-05-19T08:46:50Z")

</div>

> [@LaurentPlagne](#):
>
> The problem of doing it globally via the internal definition of Julia is that it will waste some memory space for small arrays and may reduce global performance of applications

No, I was thinking of having the pools used for small allocations set up so that things would be better aligned for their use, to avoid crossing cache lines also, for example.  
It would not waste any more space than the current scheme.

---

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [May 19, 2018, 9:16am UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/19 "2018-05-19T09:16:40Z")

</div>

OK I see, for a memory pool it should work. But if you have to deal with a large quantify of small objects, with sizes not being multiple of 16, then you will need some extra space… A kind of 1D Tetris problem 😃. This concern differs largely from mine where I want to play on very large chunks of memory and avoid to deal with nonalignment prologues.

Because of the variety of situations, I guess that in the end, a kind of control on the allocator may be interesting to have.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [May 19, 2018, 12:24pm UTC](https://discourse.julialang.org/t/aligned-julia-arrays/10993/20 "2018-05-19T12:24:56Z")

</div>

I think that enforcing 64-byte alignment on \>= 2KB objects, plus guaranteed padding to a multiple of 64 byte (so that you can’t corrupt memory by overwriting the trailing rest of the last line) would not be too expensive and be very nice for a lot of code (and I have the feeling that this is already the case, only undocumented and hence open to changes without warning). But at this point someone who actually understands the pool allocator should chime in.

Depending on how your target arch reacts to alignment failures for the instructions you use, you can then get a very simple prologue: Either nothing (tiny objects are still handled correctly, only slower, but being fast on them is not your goal anyway), or do an alignment check and punt to generic fall-backs in case of alignment failure.

[Next page](https://discourse.julialang.org/t/aligned-julia-arrays/10993.md?page=2)