### Author:  [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)
#### Post date: [October 29, 2021, 3:30pm UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/1 "2021-10-29T15:30:19Z")
I would like to know what Julia developers may have to share about Julia development on apple silicon (M1) machines. What is nice, what is wrong, is it possible to use GLMakie, multi-threading perfs and so on…
---
### Author:  [@robsmith11](https://discourse.julialang.org/u/robsmith11)
#### Post date: [October 29, 2021, 10:23pm UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/2 "2021-10-29T22:23:00Z")
As soon as native Linux is usable for daily use (should be soon since I wouldn’t need GPU acceleration), I’d like to run some Julia benchmarks. Most of my simulations can easily use many threads but don’t scale well because they saturate memory bandwidth. But at 400 GB/s, the M1 Max has 8x the memory bandwidth of my current laptop, so it should be a big speedup.
---
### Author:  [@giordano](https://discourse.julialang.org/u/giordano)
#### Post date: [October 29, 2021, 11:21pm UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/3 "2021-10-29T23:21:36Z")
> [@LaurentPlagne](#):
>
> What is nice
Building Julia from scratch takes 4 minutes total! When it works, it’s very snappy.
> [@LaurentPlagne](#):
>
> what is wrong
- [https://github.com/JuliaLang/julia/issues/41440](https://github.com/JuliaLang/julia/issues/41440)
- [Darwin/ARM64: Julia freezes on nested `@threads` loops · Issue #41820 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/41820)
and the other issues labelled with [apple silicon](https://github.com/JuliaLang/julia/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3A%22apple+silicon%22).
> [@LaurentPlagne](#):
>
> is it possible to use GLMakie
Yes, I just tried
>
and worked correctly. Timing:
```julia
julia> @time begin
using GLMakie
function sphere(n)
u = range(0, stop=2*π, length=n)
v = range(0, stop=π, length=n)
x = sin.(u) * sin.(v)'
y = cos.(u) * sin.(v)'
z = ones(n) * cos.(v)'
return (x,y,z)
end
(X,Y,Z) = sphere(201)
R = 1 .- (1 .- mod.(0:0.1:20,2)) .^ 2/15
RX = R .* X
RY = R .* Y
RZ = (0.8 .+ (0 .- (1:-0.01:-1)' .^ 4) * 0.2) .* Z .* R
scene = Scene(show_axis=false)
surface!(scene, RX, RY, RZ, color=fill("#ff7518",1,1))
surface!(scene, X/12, Y/12, Z/2 .+ 0.4, color=fill("#080",1,1))
display(scene)
end
14.681251 seconds (67.44 M allocations: 3.864 GiB, 5.15% gc time, 69.73% compilation time)
```
> [@LaurentPlagne](#):
>
> multi-threading perfs
Good, when it doesn’t deadlock. Using more than 4 threads may not be beneficial though (at least with the original M1)
---
### Author:  [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)
#### Post date: [October 30, 2021, 8:03am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/4 "2021-10-30T08:03:54Z")
This is exactly why I am interested by this machine that overcomes my natural reluctance to closed OS as a Linux user. The (announced) CPU memory bandwidth is 20x faster than what I measure on my laptop (20GBs). It may be even more impressive considering perf/watt if this kind of perf can be achieved on a cool and QUIET laptop…
As you say, a large fraction of scientific computing kernels are memory bound (e.g. GMG, SpMV,…) and the potential of this architecture for SC looks amazing (HPCG Benchmark, Green500).
IMO, the other big point of interest is the presumably low latency CPU/GPU interoperability allowed by the SOC design. It may open GPGPU on a new class of problems previously eliminated by the CPU-GPU communication overheads. Looks like a super sweet spot for oneAPI/SYCL xPU programming.
If these points are confirmed, I guess that apple silicon strategy will drive the evolution of the competition (perf/watt being the key factor).
---
### Author:  [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)
#### Post date: [October 30, 2021, 8:20am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/5 "2021-10-30T08:20:54Z")
Thank you very much for your explanations !
I guess that, solving this issues may improve the robustness and quality of Julia’s implementation.
In particular, the threading issues may reveal weak assumptions and help to improve the heterogeneous cores management that seems to become the new standard (Alder Lake ?).
> [@giordano](#):
>
> Good, when it doesn’t deadlock.
This quote is a bit terrifying 🙂
> [@giordano](#):
>
> Using more than 4 threads may not be beneficial though (at least with the original M1)
… the 4 high efficiency cores do not cooperate nicely with the 4 high performances cores…
The M1 pro/MAX have 10 perf cores : the scaling should be better 😉
---
### Author:  [@Elrod](https://discourse.julialang.org/u/Elrod)
#### Post date: [October 30, 2021, 8:30am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/6 "2021-10-30T08:30:34Z")
> [@LaurentPlagne](#):
>
> This is exactly why I am interested by this machine that overcomes my natural reluctance to closed OS as a Linux user. The (announced) CPU memory bandwidth is 20x faster than what I measure on my laptop (20GBs).
I just tried [STREAMBenchmark.jl](https://github.com/JuliaPerf/STREAMBenchmark.jl/pull/14) on my M1 and got 90 GB/s for some benchmarks on a single thread. Multithreading does not improve performance.
I too prefer (and primarily use) Linux, but I like to have the M1 around for benchmarking.
While it won’t be available in laptop chips, Intel’s upcoming Saphire Rapids will offer some chips with HBM, and [some sources](https://www.anandtech.com/show/16795/intel-to-launch-next-gen-sapphire-rapids-xeon-with-high-bandwidth-memory) are speculating 1TB/s or so memory bandwidth (divided among many more cores, of course).
AMD’s 3d stacking/V-Cache will give many of their chips a very large L3 cache (which itself [could have 2TB/s bandwidth](https://www.extremetech.com/computing/326194-amd-unveils-new-ryzen-v-cache-details-at-hotchips-33) per chiplet, but at 32+64 MiB is much smaller than HBM modules), which (depending on the workload) could help a great deal as well.
So still some interesting developments in x86/Linux compatible land coming in the next year.
> The M1 pro/MAX have 10 perf cores
8 perf + 2 efficiency.
I may just need to test them more to get threading deadlocks, but I haven’t seen them from LoopVectorization/Polyester. My impression (having not investigated it much) is that base threading and libraries using it are at risk, particularly in code that spawns tasks relatively rapidly.
---
### Author:  [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)
#### Post date: [October 30, 2021, 8:39am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/7 "2021-10-30T08:39:28Z")
Hi @Elrod , we should ship in to get you the M1 Max model 😉
You are comparing future server architectures with an available laptop : again, I think that the perf/watt (or bandwidth/watt) is the most relevant metric to consider.
---
### Author:  [@giordano](https://discourse.julialang.org/u/giordano)
#### Post date: [October 30, 2021, 8:39am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/8 "2021-10-30T08:39:36Z")
Looks like I have been too optimistic about Makie:
- [https://github.com/JuliaPlots/GraphMakie.jl/issues/34](https://github.com/JuliaPlots/GraphMakie.jl/issues/34)
- [https://github.com/JuliaLang/julia/issues/42624](https://github.com/JuliaLang/julia/issues/42624)
These appear to be all related to [https://github.com/JuliaLang/julia/issues/41440](https://github.com/JuliaLang/julia/issues/41440). There is a path forward to address the problem, but someone has to do the work.
---
### Author:  [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)
#### Post date: [October 30, 2021, 8:42am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/9 "2021-10-30T08:42:51Z")
Is it a a problem with Julia implementation or a LLVM bug ?
---
### Author:  [@Elrod](https://discourse.julialang.org/u/Elrod)
#### Post date: [October 30, 2021, 8:42am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/10 "2021-10-30T08:42:59Z")
Yeah, unfortunately there doesn’t seem to be any competition there at the moment.
#41440 problems are pretty frequent at the moment.
Personally, I assumed that the threading deadlocks aren’t due to heterogenous cores, but the weaker memory model of ARM (vs x86) + the massive out of order of the M1 exposing bugs in the threading implementation.
The M1 doesn’t deadlock when running under Rosetta (emulating x86), for example. When doing so, it uses the x86 memory model.
---
### Author:  [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)
#### Post date: [October 30, 2021, 9:02am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/11 "2021-10-30T09:02:57Z")
I see, your referring to addition of barriers like in these threads
> **[C++ Memory Model: Migrating from X86 to ARM](https://www.arangodb.com/2021/02/cpp-memory-model-migrating-from-x86-to-arm/)**
>
> Over the past years, we have seen a shift in processors from the previously dominant x86 architecture to the more energy-efficient (and often cheaper) ARM architecture. This trend is true for both consumer hardware, e.g., Apple’s M1 SoC and also...
> **[The Apple M1](https://arstechnica.com/civis/threads/the-apple-m1.1472085/page-6)**
>
> Relevant: ARM and Lock-Free Programming
>
> Tldr: "It turns out that ARM’s weak memory model really doesn’t make things any more complicated. If you are writing lock-free code and not using any sort of memory barriers then your code is potentially...
Complex stuff… slightly above my head 😊
---
### Author:  [@giordano](https://discourse.julialang.org/u/giordano)
#### Post date: [October 30, 2021, 9:03am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/12 "2021-10-30T09:03:02Z")
It’s a bug on Julia’s side, wrong code model used, see [https://github.com/JuliaLang/julia/issues/41440#issuecomment-932048448](https://github.com/JuliaLang/julia/issues/41440#issuecomment-932048448) and following messages.
---
### Author:  [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)
#### Post date: [October 30, 2021, 10:00am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/13 "2021-10-30T10:00:23Z")
Do threads checker tools as [Detect Data Races Among Your App’s Threads](https://developer.apple.com/documentation/xcode/diagnosing-memory-thread-and-crash-issues-early) help to catch some bugs ?
---
### Author:  [@Storopoli](https://discourse.julialang.org/u/Storopoli)
#### Post date: [October 30, 2021, 10:18am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/14 "2021-10-30T10:18:43Z")
What about LibTask?
I use `Turing.jl` a lot in almost all of my research and Julia code. I have a M1 MacBook Air but I haven’t yet installed the `1.7-rc`s because I was getting errors on installing `Turing.jl` in the `1.7-beta`s.
---
### Author:  [@giordano](https://discourse.julialang.org/u/giordano)
#### Post date: [October 30, 2021, 10:40am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/15 "2021-10-30T10:40:42Z")
> [@Storopoli](#):
>
> What about LibTask?
It’s stalling
>
>
> Not Ready to merge yet, now only for reviewing and discussing.
But I’m still not convinced that having to build a binary library for that is a good idea 🤷♂️
>
>
> Maintaining a library like \`Libtask\_jll\` which links to libjulia isn't super sim…ple as you need to have libjulia\_jll available in the first place and then build against all different variations. Why don't you \`ccall\` directly into libjulia from Julia, instead of delegating the same work to an external library? This would have the advantage of having \`Libtask.jl\` more readily available on all platforms and versions of Julia without depending on external libraries.
---
### Author:  [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)
#### Post date: [October 31, 2021, 8:43am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/16 "2021-10-31T08:43:39Z")
If your assumption is correct, should this weaker memory model also affect Julia execution when running on a linux parallel VM ?
> [@Julia on M1 Macs](https://discourse.julialang.org/t/julia-on-m1-macs/55448/6):
>
> Also worth noting is that you can build Julia for Linux Arm from src on a Ubuntu for Arm virtual machine running on Parallels Desktop on the M1. Its performance is actually better than Julia on Rosetta 2 for me.
---
### Author:  [@Elrod](https://discourse.julialang.org/u/Elrod)
#### Post date: [November 1, 2021, 5:53am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/17 "2021-11-01T05:53:26Z")
Hmm.
I tried running the second example from [the issue Mose linked](https://github.com/JuliaLang/julia/issues/41820).
It hung when run natively, but not when run on a Linux (AArch64) VM.
The Linux VM shouldn’t be hitting the segfaults either, so seems like that’ll be the way to go.
---
### Author:  [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)
#### Post date: [November 1, 2021, 8:37am UTC](https://discourse.julialang.org/t/apple-m1-m1-pro-m1-max-and-julia-developpers/70627/18 "2021-11-01T08:37:39Z")
> [@Elrod](#):
>
> The Linux VM shouldn’t be hitting the segfaults either, so seems like that’ll be the way to go.
You mean that I should use a linux VM on apple silicon while these issues are not fixed ?
If
- the Julia implementation (the C++ part)
- the target architecture of clang
- clang itself
are all the same, what is the difference ? OS threads management ?
> [@Elrod](#):
>
> I tried running the second example from [the issue Mose linked](https://github.com/JuliaLang/julia/issues/41820).
> It hung when run natively, but not when run on a Linux (AArch64) VM.
But the weaker ARM’s memory model should be the same on Linux AArch64 VM (no Rosetta 2 translation in this case) or I am missing something ?
It is surprising that such a small M(N)WE like this [Darwin/ARM64: Julia freezes on nested `@threads` loops · Issue #41820 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/41820#issuecomment-899175606) brings a bug so difficult to catch. Is the thread sanitizer clang option (`-fsanitize=thread`) totally useless in this case ? Are the generated machine codes very different between native and inside the ARM VM ?
OK, I realize that all these questions are probably irrelevant and may be boring coming from an outsider like me and that I should first get a M1 machine to try to catch up what is being investigated for several months now. Anyway, thank you again @Elrod and @giordano for all your explanations !