# C routine uses AVX intrinsics

**URL:** <https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777>\
**Category:** General Usage\
**Tags:** interoperability, c\
**Created:** [September 25, 2022, 10:51am UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777 "2022-09-25T10:51:10Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [September 25, 2022, 10:51am UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/1 "2022-09-25T10:51:10Z")

</div>

I would like to make the example [in the docs](https://docs.julialang.org/en/v1.9-dev/manual/calling-c-and-fortran-code/#SIMD-Values) work. The presentation is incomplete; while I have the requisite tools, it is unclear how best (most simply and effectively) apply them here: the easiest most foolproof way to create a file that does the C and allows it to be part of the distribution of a package. I understand this is a machine specific facility. It would be useful to do the same with similar C++.

```julia
#include <immintrin.h>

__m256 dist(__m256 a, __m256 b ) {
    return _mm256_sqrt_ps(
              _mm256_add_ps(_mm256_mul_ps(a, a),
                            _mm256_mul_ps(b, b)));
}

```

```julia
const m256 = NTuple{8, VecElement{Float32}}

a = m256(ntuple(i -> VecElement(sin(Float32(i))), 8))
b = m256(ntuple(i -> VecElement(cos(Float32(i))), 8))

function call_dist(a::m256, b::m256)
    @ccall "libdist".dist(a::m256, b::m256)::m256
end

println(call_dist(a,b))

```

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [September 25, 2022, 11:03am UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/2 "2022-09-25T11:03:30Z")

</div>

To be sure, I would rather do this in Julia directly. While we have a bunch of accessible and utilitarian AVX2 acceleration, there are areas of available SIMD instructions that have not been well developed with Julia (as yet). This is not a gripe – the priorities have been sound. I need finer control of some of the other SIMD instructions to improve the throughput of error-free transformations. That would benefit Double64s.

---

<div class="post-metadata">

**Author:** ![woclass](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/woclass/32/212699_2.png) [@woclass](https://discourse.julialang.org/u/woclass)\
**Post date:** [September 25, 2022, 11:09am UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/3 "2022-09-25T11:09:49Z")

</div>

We may want to rearrange the order of the documents.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/1/8/188bb1f59ac12d88e0d031c2d4575826bfac8b11.png)

Compile you code:

```sh
gcc -mavx -fPIC -shared -o libdist.so avx.c

```

open julia in the same dir:

```julia
julia> pwd()
"/mnt/d/proj/Julia/JuliaEn/87777-ccall"

shell> ls
avx.c libdist.so

julia> const m256 = NTuple{8, VecElement{Float32}}
NTuple{8, VecElement{Float32}}

julia> a = m256(ntuple(i -> VecElement(sin(Float32(i))), 8))
(VecElement{Float32}(0.84147096f0), VecElement{Float32}(0.9092974f0), VecElement{Float32}(0.14112f0), VecElement{Float32}(-0.7568025f0), VecElement{Float32}(-0.9589243f0), VecElement{Float32}(-0.2794155f0), VecElement{Float32}(0.6569866f0), VecElement{Float32}(0.98935825f0))

julia> b = m256(ntuple(i -> VecElement(cos(Float32(i))), 8))
(VecElement{Float32}(0.5403023f0), VecElement{Float32}(-0.41614684f0), VecElement{Float32}(-0.9899925f0), VecElement{Float32}(-0.6536436f0), VecElement{Float32}(0.2836622f0), VecElement{Float32}(0.96017027f0), VecElement{Float32}(0.75390226f0), VecElement{Float32}(-0.14550003f0))

julia> function call_dist(a::m256, b::m256)
           @ccall "./libdist".dist(a::m256, b::m256)::m256
       end
call_dist (generic function with 1 method)

julia> println(call_dist(a,b))
(VecElement{Float32}(0.99999994f0), VecElement{Float32}(0.99999994f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0))

```

Note: I’m using `"./libdist"` to call the lib.  
Calling the library directly using the name will give an error: “ERROR: could not load library “libdist””.  
Maybe I need to set some environment variables.

* * *

To load the shared libraries under the current path, you need to modify `DL_LOAD_PATH`: `push!(DL_LOAD_PATH, ".")`

```julia
julia> ccall((:dist, "libdist"), (m256), (m256,m256), a,b)
ERROR: could not load library "libdist"
libdist.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] top-level scope
   @ ./REPL[36]:1

julia> push!(DL_LOAD_PATH, ".")
1-element Vector{String}:
 "."

julia> ccall((:dist, "libdist"), (m256), (m256,m256), a,b)
(VecElement{Float32}(0.99999994f0), VecElement{Float32}(0.99999994f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0), VecElement{Float32}(1.0f0))

```

---

<div class="post-metadata">

**Author:** ![woclass](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/woclass/32/212699_2.png) [@woclass](https://discourse.julialang.org/u/woclass)\
**Post date:** [September 25, 2022, 11:23am UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/4 "2022-09-25T11:23:54Z")

</div>

Next, you may want to use [GitHub - JuliaPackaging/BinaryBuilder.jl: Binary Dependency Builder for Julia](https://github.com/JuliaPackaging/BinaryBuilder.jl) to automatically compile the C libraries for each platform and package them as jll package.

Then you can simply `using LibDist_jll.jl` in julia.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [September 25, 2022, 11:53am UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/5 "2022-09-25T11:53:23Z")

</div>

The ccall will not be inclined, so performance will be bad use `llvmcall` if you want performance.

You have access to everything intrinsic Clang is capable of that way.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [September 25, 2022, 11:56am UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/6 "2022-09-25T11:56:19Z")

</div>

It is too early for me to say how well it works 🙂

Is there something equally graspable that would allow me to make [google’s highway](https://github.com/google/highway) [portable SIMD/vector intrinsics in C++] available? Or any similar facility.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [September 25, 2022, 11:59am UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/7 "2022-09-25T11:59:44Z")

</div>

Here is an example of calling an instruction from a particular instruction set: [VectorizationBase.jl/conflict.jl at master · JuliaSIMD/VectorizationBase.jl · GitHub](https://github.com/JuliaSIMD/VectorizationBase.jl/blob/master/src/llvm_intrin/conflict.jl)

If you want portability, don’t call generic llvm Vec for instructions. LLVM does a great job. Better than highway.  
Julia thankfully exposes that to users. Clang unfortunately does not.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [September 25, 2022, 12:18pm UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/8 "2022-09-25T12:18:55Z")

</div>

good information – thanks

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 25, 2022, 1:39pm UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/9 "2022-09-25T13:39:42Z")

</div>

Using Godbolt to emit LLVM IR:

> **[Compiler Explorer - C++ (x86-64 clang 14.0.0)](https://godbolt.org/z/PrnadKdv7)**
>
> \_\_m256 dist( \_\_m256 a, \_\_m256 b ) {
> return \_mm256\_sqrt\_ps(
> \_mm256\_add\_ps(\_mm256\_mul\_ps(a, a),
> \_mm256\_mul\_ps(b, b)));
> }

```llvm

define dso_local noundef <8 x float> @_Z4distDv8_fS_(<8 x float> noundef %0, <8 x float> noundef %1) #0 !dbg !261 {
  %3 = alloca <8 x float>, align 32
  %4 = alloca <8 x float>, align 32
  %5 = alloca <8 x float>, align 32
  %6 = alloca <8 x float>, align 32
  %7 = alloca <8 x float>, align 32
  %8 = alloca <8 x float>, align 32
  %9 = alloca <8 x float>, align 32
  %10 = alloca <8 x float>, align 32
  %11 = alloca <8 x float>, align 32
  store <8 x float> %0, <8 x float>* %10, align 32
  call void @llvm.dbg.declare(metadata <8 x float>* %10, metadata !266, metadata !DIExpression()), !dbg !267
  store <8 x float> %1, <8 x float>* %11, align 32
  call void @llvm.dbg.declare(metadata <8 x float>* %11, metadata !268, metadata !DIExpression()), !dbg !269
  %12 = load <8 x float>, <8 x float>* %10, align 32, !dbg !270
  %13 = load <8 x float>, <8 x float>* %10, align 32, !dbg !271
  store <8 x float> %12, <8 x float>* %3, align 32, !dbg !272
  store <8 x float> %13, <8 x float>* %4, align 32, !dbg !272
  %14 = load <8 x float>, <8 x float>* %3, align 32, !dbg !272
  %15 = load <8 x float>, <8 x float>* %4, align 32, !dbg !272
  %16 = fmul <8 x float> %14, %15, !dbg !272
  %17 = load <8 x float>, <8 x float>* %11, align 32, !dbg !273
  %18 = load <8 x float>, <8 x float>* %11, align 32, !dbg !274
  store <8 x float> %17, <8 x float>* %5, align 32, !dbg !275
  store <8 x float> %18, <8 x float>* %6, align 32, !dbg !275
  %19 = load <8 x float>, <8 x float>* %5, align 32, !dbg !275
  %20 = load <8 x float>, <8 x float>* %6, align 32, !dbg !275
  %21 = fmul <8 x float> %19, %20, !dbg !275
  store <8 x float> %16, <8 x float>* %7, align 32, !dbg !276
  store <8 x float> %21, <8 x float>* %8, align 32, !dbg !276
  %22 = load <8 x float>, <8 x float>* %7, align 32, !dbg !276
  %23 = load <8 x float>, <8 x float>* %8, align 32, !dbg !276
  %24 = fadd <8 x float> %22, %23, !dbg !276
  store <8 x float> %24, <8 x float>* %9, align 32, !dbg !277
  %25 = load <8 x float>, <8 x float>* %9, align 32, !dbg !277
  %26 = call <8 x float> @llvm.sqrt.v8f32(<8 x float> %25) #2, !dbg !277
  ret <8 x float> %26, !dbg !278
}

declare void @llvm.dbg.declare(metadata, metadata, metadata) #1

declare <8 x float> @llvm.sqrt.v8f32(<8 x float>) #1

attributes #0 = { mustprogress noinline optnone uwtable "frame-pointer"="all" "min-legal-vector-width"="256" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+avx,+avx2,+crc32,+cx8,+fxsr,+mmx,+popcnt,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+x87,+xsave" "tune-cpu"="generic" }
attributes #1 = { nofree nosync nounwind readnone speculatable willreturn }
attributes #2 = { nounwind }

```

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [September 25, 2022, 5:06pm UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/10 "2022-09-25T17:06:40Z")

</div>

How do I use the emitted instructions, massage them into julia?

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [September 25, 2022, 5:37pm UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/11 "2022-09-25T17:37:47Z")

</div>

You could `llvmcall` them.  
But you should already be able to see that it isn’t doing anything SIMD.jl or VectorizationBase.jl aren’t already doing. They’ve both already wrapped `llvmcall` you’d need (you don’t need the `alloca`s, and while they have wrapped the `load` and `store`s, which are useful for many things, they’re not needed here either).

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 25, 2022, 7:37pm UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/12 "2022-09-25T19:37:43Z")

</div>

@kristoffer.carlsson wrote a blog post on how this could be done.

Once you know the LLVM intrinsics of interest you could look in thr following files and see id they have wrapped for you already.

> <https://github.com/eschnett/SIMD.jl/blob/master/src/LLVM_intrinsics.jl>

> **[VectorizationBase.jl/src/llvm\_intrin at master · JuliaSIMD/VectorizationBase.jl](https://github.com/JuliaSIMD/VectorizationBase.jl/tree/master/src%2Fllvm_intrin)**
>
> master/src%2Fllvm\_intrin

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 25, 2022, 8:21pm UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/13 "2022-09-25T20:21:18Z")

</div>

Let’s see if I can help map this out.

# Intel Intrinsics

1. [`_mm256_mul_ps`](https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm256_mul_ps)
2. [`_mm256_add_ps`](https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm256_add_ps&ig_expand=4960,156)
3. [`_mm256_sqrt_ps`](https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm256_sqrt_ps&ig_expand=4960,156,6614)

The gist is that we are trying to work on 8 `Float32`s at a time.

# LLVM IR to SIMD.jl and VectorizationBase.jl

1. [`load`](https://llvm.org/docs/LangRef.html?highlight=trunc#load-instruction)
  - [`SIMD.Intrinsics.load`](https://github.com/eschnett/SIMD.jl/blob/1555361e8a3c9ee2b3ff395773e5dddc3e48edc6/src/LLVM_intrinsics.jl#L442)
  - `VectorizationBase.vload`

2. [`store`](https://llvm.org/docs/LangRef.html?highlight=trunc#store-instruction)
  - [`SIMD.Intrinsics.store`](https://github.com/eschnett/SIMD.jl/blob/1555361e8a3c9ee2b3ff395773e5dddc3e48edc6/src/LLVM_intrinsics.jl#L477)
  - `VectorizationBase.vstore!`

3. [`fmul`](https://llvm.org/docs/LangRef.html?highlight=trunc#fmul-instruction)
  - [`SIMD.Intrinsics.fmul`](https://github.com/eschnett/SIMD.jl/blob/1555361e8a3c9ee2b3ff395773e5dddc3e48edc6/src/LLVM_intrinsics.jl#L190)
  - [`VectorizationBase.vmul`](https://github.com/JuliaSIMD/VectorizationBase.jl/blob/9ae47c5c12c7bec9b57c132c24f67c4f14090c9e/src/llvm_intrin/binary_ops.jl#L100)

4. [`fadd`](https://llvm.org/docs/LangRef.html?highlight=trunc#fadd-instruction)
  - [`SIMD.Intrinsics.fadd`](https://github.com/eschnett/SIMD.jl/blob/1555361e8a3c9ee2b3ff395773e5dddc3e48edc6/src/LLVM_intrinsics.jl#L188)
  - [`VectorizationBase.vadd`](https://github.com/JuliaSIMD/VectorizationBase.jl/blob/9ae47c5c12c7bec9b57c132c24f67c4f14090c9e/src/llvm_intrin/binary_ops.jl#L100)

5. [`llvm.sqrt`](https://llvm.org/docs/LangRef.html?highlight=trunc#llvm-sqrt-intrinsic)
  - [`SIMD.Intrinsics.sqrt`](https://github.com/eschnett/SIMD.jl/blob/1555361e8a3c9ee2b3ff395773e5dddc3e48edc6/src/LLVM_intrinsics.jl#L93)
  - [`VectorizationBase.vsqrt`](https://github.com/JuliaSIMD/VectorizationBase.jl/blob/5076439ec997498e262de078bd3ebfbd2775d561/src/llvm_intrin/intrin_funcs.jl#L130)

Essentially have all the pieces. Now to figure out how to assemble them.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [September 25, 2022, 8:22pm UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/14 "2022-09-25T20:22:31Z")

</div>

Thank you for the gracious walkway.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 25, 2022, 9:03pm UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/15 "2022-09-25T21:03:16Z")

</div>

Here is the SIMD.jl version:

```julia
julia> function dist(a::NTuple{8,Float32}, b::NTuple{8,Float32})::NTuple{8,Float32}
           va = Vec{8,Float32}(a)
           vb = Vec{8,Float32}(b)
           vc = sqrt(va^2 + vb^2)
           return Tuple(vc)
       end
dist (generic function with 1 method)

julia> @code_llvm debuginfo=:none dist(a,b)
define void @julia_dist_508([8 x float]* noalias nocapture sret([8 x float]) %0, [8 x float]* nocapture nonnull readonly align 4 dereferenceable(32) %1, [8 x float]* nocapture nonnull readonly align 4 dereferenceable(32) %2) #0 {
top:
  %3 = bitcast [8 x float]* %1 to <8 x float>*
  %4 = load <8 x float>, <8 x float>* %3, align 4
  %5 = bitcast [8 x float]* %2 to <8 x float>*
  %6 = load <8 x float>, <8 x float>* %5, align 4
  %7 = fmul <8 x float> %4, %4
  %8 = fmul <8 x float> %6, %6
  %9 = fadd <8 x float> %7, %8
  %10 = call <8 x float> @llvm.sqrt.v8f32(<8 x float> %9)
  %11 = bitcast [8 x float]* %0 to <8 x float>*
  store <8 x float> %10, <8 x float>* %11, align 4
  ret void
}

julia> @code_native debuginfo=:none dist(a,b)
	.text
	.file	"dist"
	.globl	julia_dist_513 # -- Begin function julia_dist_513
	.p2align	4, 0x90
	.type	julia_dist_513,@function
julia_dist_513: # @julia_dist_513
	.cfi_startproc
# %bb.0: # %top
	vmovups	(%rsi), %ymm0
	vmovups	(%rdx), %ymm1
	movq	%rdi, %rax
	vmulps	%ymm0, %ymm0, %ymm0
	vmulps	%ymm1, %ymm1, %ymm1
	vaddps	%ymm1, %ymm0, %ymm0
	vsqrtps	%ymm0, %ymm0
	vmovups	%ymm0, (%rdi)
	vzeroupper
	retq
.Lfunc_end0:
	.size	julia_dist_513, .Lfunc_end0-julia_dist_513
	.cfi_endproc
                                        # -- End function
	.section	".note.GNU-stack","",@progbits

```

That looks similar to what the C version compiles to on x86\_64:

```nohighlight
dist(float __vector(8), float__vector(8)): # @dist(float __vector(8), float__vector(8))
        push rbp
        mov rbp, rsp
        and rsp, -32
        sub rsp, 320
        vmovaps ymmword ptr [rsp + 32], ymm0
        vmovaps ymmword ptr [rsp], ymm1
        vmovaps ymm1, ymmword ptr [rsp + 32]
        vmovaps ymm0, ymmword ptr [rsp + 32]
        vmovaps ymmword ptr [rsp + 256], ymm1
        vmovaps ymmword ptr [rsp + 224], ymm0
        vmovaps ymm0, ymmword ptr [rsp + 256]
        vmulps ymm1, ymm0, ymmword ptr [rsp + 224]
        vmovaps ymm2, ymmword ptr [rsp]
        vmovaps ymm0, ymmword ptr [rsp]
        vmovaps ymmword ptr [rsp + 192], ymm2
        vmovaps ymmword ptr [rsp + 160], ymm0
        vmovaps ymm0, ymmword ptr [rsp + 192]
        vmulps ymm0, ymm0, ymmword ptr [rsp + 160]
        vmovaps ymmword ptr [rsp + 128], ymm1
        vmovaps ymmword ptr [rsp + 96], ymm0
        vmovaps ymm0, ymmword ptr [rsp + 128]
        vaddps ymm0, ymm0, ymmword ptr [rsp + 96]
        vmovaps ymmword ptr [rsp + 64], ymm0
        vmovaps ymm0, ymmword ptr [rsp + 64]
        vsqrtps ymm0, ymm0
        mov rsp, rbp
        pop rbp
        ret

```

[Godbolt link to assembly](https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(filename:'1',fontScale:14,fontUsePx:'0',j:1,lang:c%2B%2B,selection:(endColumn:42,endLineNumber:5,positionColumn:42,positionLineNumber:5,selectionStartColumn:29,selectionStartLineNumber:5,startColumn:29,startLineNumber:5),source:'%23include+%3Cimmintrin.h%3E%0A%0A__m256+dist(+__m256+a,+__m256+b+)+%7B%0A++++return+_mm256_sqrt_ps(%0A++++++++++++++_mm256_add_ps(_mm256_mul_ps(a,+a),%0A++++++++++++++++++++++++++++_mm256_mul_ps(b,+b)))%3B%0A%7D%0A'),l:'5',n:'0',o:'C%2B%2B+source+%231',t:'0')),k:50,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:compiler,i:(compiler:clang1400,filters:(b:'0',binary:'1',commentOnly:'0',demangle:'0',directives:'0',execute:'1',intel:'0',libraryCode:'0',trim:'1'),flagsViewOpen:'1',fontScale:14,fontUsePx:'0',j:1,lang:c%2B%2B,libs:!(),options:'-mavx2',selection:(endColumn:12,endLineNumber:29,positionColumn:1,positionLineNumber:1,selectionStartColumn:12,selectionStartLineNumber:29,startColumn:1,startLineNumber:1),source:1,tree:'1'),l:'5',n:'0',o:'x86-64+clang+14.0.0+(C%2B%2B,+Editor+%231,+Compiler+%231)',t:'0')),k:50,l:'4',n:'0',o:'',s:0,t:'0')),l:'2',n:'0',o:'',t:'0')),version:4)

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 26, 2022, 9:19pm UTC](https://discourse.julialang.org/t/c-routine-uses-avx-intrinsics/87777/17 "2022-09-26T21:19:33Z")

</div>

I also did this with VectorizationBase.jl on another computer with a Xeon as an exercise for myself. Admittedly, we could use some improved documentation there, but perhaps it is still not meant to be used directly?

```julia
julia> using VectorizationBase

julia> dist(a::NTuple{N,T}, b::NTuple{N,T}) where {N,T} = return ntuple(sqrt(Vec(a...)^2 + Vec(b...)^2), Val(N))
dist (generic function with 2 methods)

julia> a = b = (1.f0,2.f0,3.f0,4.f0,5.f0,6.f0,7.f0,8.f0)
(1.0f0, 2.0f0, 3.0f0, 4.0f0, 5.0f0, 6.0f0, 7.0f0, 8.0f0)

julia> @code_llvm debuginfo=:none dist(a,b)
; Function Attrs: uwtable
define void @julia_dist_1430([8 x float]* noalias nocapture sret([8 x float]) %0, [8 x float]* nocapture nonnull readonly align 4 dereferenceable(32) %1, [8 x float]* nocapture nonnull readonly align 4 dereferenceable(32) %2) #0 {
top:
  %3 = bitcast [8 x float]* %1 to <8 x float>*
  %4 = load <8 x float>, <8 x float>* %3, align 4
  %res.i26 = fmul reassoc nsz arcp contract afn <8 x float> %4, %4
  %5 = bitcast [8 x float]* %2 to <8 x float>*
  %6 = load <8 x float>, <8 x float>* %5, align 4
  %res.i17 = fmul reassoc nsz arcp contract afn <8 x float> %6, %6
  %res.i16 = fadd nsz contract <8 x float> %res.i26, %res.i17
  %res.i15 = call fast <8 x float> @llvm.sqrt.v8f32(<8 x float> %res.i16)
  %7 = bitcast [8 x float]* %0 to <8 x float>*
  store <8 x float> %res.i15, <8 x float>* %7, align 4
  ret void
}

julia> @code_native debuginfo=:none dist(a,b)
        .text
        .file "dist"
        .globl julia_dist_1446 # -- Begin function julia_dist_1446
        .p2align 4, 0x90
        .type julia_dist_1446,@function
julia_dist_1446: # @julia_dist_1446
        .cfi_startproc
# %bb.0: # %top
        pushq %rbp
        .cfi_def_cfa_offset 16
        .cfi_offset %rbp, -16
        movq %rsp, %rbp
        .cfi_def_cfa_register %rbp
        vmovups (%rdx), %ymm0
        vmovups (%r8), %ymm1
        movq %rcx, %rax
        vmulps %ymm1, %ymm1, %ymm1
        vfmadd231ps %ymm0, %ymm0, %ymm1 # ymm1 = (ymm0 * ymm0) + ymm1
        vsqrtps %ymm1, %ymm0
        vmovups %ymm0, (%rcx)
        popq %rbp
        vzeroupper
        retq
.Lfunc_end0:
        .size julia_dist_1446, .Lfunc_end0-julia_dist_1446
        .cfi_endproc
                                        # -- End function
        .section ".note.GNU-stack","",@progbits

```

What is impressive here is that the compiler chose an advanced instruction,`vfmadd231ps` which is a fused multiply add. The great thing about Julia is that it can adapt to the architecture it is being used on easily. Here it is using AVX512 instruction:

[https://www.felixcloutier.com/x86/vfmadd132ps:vfmadd213ps:vfmadd231ps](https://www.felixcloutier.com/x86/vfmadd132ps:vfmadd213ps:vfmadd231ps)

This version can also be easily adapted to use 512-bit registers (see the `zmm*`):

```julia
julia> a = b = (1, 2, 3, 4, 5, 6, 7, 8)
(1, 2, 3, 4, 5, 6, 7, 8)

julia> @code_native debuginfo=:none dist(a,b)
        .text
        .file "dist"
        .globl julia_dist_1521 # -- Begin function julia_dist_1521
        .p2align 4, 0x90
        .type julia_dist_1521,@function
julia_dist_1521: # @julia_dist_1521
        .cfi_startproc
# %bb.0: # %top
        pushq %rbp
        .cfi_def_cfa_offset 16
        .cfi_offset %rbp, -16
        movq %rsp, %rbp
        .cfi_def_cfa_register %rbp
        vcvtqq2pd (%rdx), %zmm0
        vcvtqq2pd (%r8), %zmm1
        vmulpd %zmm1, %zmm1, %zmm1
        vfmadd231pd %zmm0, %zmm0, %zmm1 # zmm1 = (zmm0 * zmm0) + zmm1
        vsqrtpd %zmm1, %zmm0
        movq %rcx, %rax
        vmovupd %ymm0, (%rcx)
        vextractf64x4 $1, %zmm0, 32(%rcx)
        popq %rbp
        vzeroupper
        retq
.Lfunc_end0:
        .size julia_dist_1521, .Lfunc_end0-julia_dist_1521
        .cfi_endproc
                                        # -- End function
        .section ".note.GNU-stack","",@progbits

```