# Julia access to Apple GPU with MLX, and or Metal Performance Shaders (MPS)?

**URL:** <https://discourse.julialang.org/t/julia-access-to-apple-gpu-with-mlx-and-or-metal-performance-shaders-mps/117647>\
**Category:** Performance\
**Created:** [July 30, 2024, 7:20pm UTC](https://discourse.julialang.org/t/julia-access-to-apple-gpu-with-mlx-and-or-metal-performance-shaders-mps/117647 "2024-07-30T19:20:32Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![pitsianis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pitsianis/32/26588_2.png) [@pitsianis](https://discourse.julialang.org/u/pitsianis)\
**Post date:** [July 30, 2024, 7:20pm UTC](https://discourse.julialang.org/t/julia-access-to-apple-gpu-with-mlx-and-or-metal-performance-shaders-mps/117647/1 "2024-07-30T19:20:33Z")

</div>

Is there any on-going effort to provide a Julia interface to the following?

- [Metal Performance Shaders (MPS)](https://developer.apple.com/metal/pytorch/) is a PyTorch framework backend for GPU training acceleration, providing scripts and capabilities to set up and run operations on Mac.

- [MLX](https://github.com/ml-explore/mlx) is an array framework for machine learning research on Apple silicon, by Apple machine learning research. MLX provides FFTs, Linear Algebra operations Cholesky, inv, qr, svd, and includes a list of of examples:

Or calling them via their Python interfaces using `PythonCall.jl` is the only (good-enough?) way? MLX has a C API [MLX-C](https://github.com/ml-explore/mlx-c)

Apologies if this has been discussed already, but I could not locate any related information here.

---

<div class="post-metadata">

**Author:** ![stemann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stemann/32/4030_2.png) [@stemann](https://discourse.julialang.org/u/stemann)\
**Post date:** [November 6, 2024, 11:22am UTC](https://discourse.julialang.org/t/julia-access-to-apple-gpu-with-mlx-and-or-metal-performance-shaders-mps/117647/2 "2024-11-06T11:22:31Z")

</div>

MPS is not a PyTorch Framework. MPS are implementations of common operations as Metal (GPU) kernels - one sub set of which is implementing NN/CNN operations. In addition, PyTorch has a MPS back-end which uses the MPS kernels to accelerate PyTorch operations etc.

MPS NN/CNN-support via Metal.jl (and NNlib.jl, and in turn Flux.jl) is so far tracked in [MPS - Support for Convolutional Neural Network kernels · Issue #210 · JuliaGPU/Metal.jl · GitHub](https://github.com/JuliaGPU/Metal.jl/issues/210).

If I am not mistaken, there has not yet been any public applications of MLX in Julia.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [November 8, 2024, 1:40am UTC](https://discourse.julialang.org/t/julia-access-to-apple-gpu-with-mlx-and-or-metal-performance-shaders-mps/117647/3 "2024-11-08T01:40:40Z")

</div>

> [@pitsianis](#):
>
> MLX has a C API [MLX-C](https://github.com/ml-explore/mlx-c)

That API seems the way to go, to wrap (with a JLL) if not already done, should be simple, you could also call without a JLL.

> [@pitsianis](#):
>
> - Large-scale text generation with [LLaMA](https://github.com/ml-explore/mlx-examples/tree/main/llms/llama) and finetuning with [LoRA](https://github.com/ml-explore/mlx-examples/tree/main/lora).
> - Generating images with [Stable Diffusion](https://github.com/ml-explore/mlx-examples/tree/main/stable_diffusion).
> - Speech recognition with [OpenAI’s Whisper](https://github.com/ml-explore/mlx-examples/tree/main/whisper).

High-level API to that would be of help, e.g. to Wisper, not just for Apple hardware, i.e. use that code there, and other alternatives elsewhere.

It’s good enough for most people calling Python with PythonCall.jl for stuff like that, though might be better to call some such (cross-platform) Rust wrapper or other, if not done in Julia.

> MLX has a Python API that closely follows NumPy. MLX also has fully featured C++, [C](https://github.com/ml-explore/mlx-c), and [Swift](https://github.com/ml-explore/mlx-swift/) APIs, which closely mirror the Python API. MLX has higher-level packages like `mlx.nn` and `mlx.optimizers` with APIs that closely follow PyTorch to simplify building more complex models.

---

<div class="post-metadata">

**Author:** ![pitsianis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pitsianis/32/26588_2.png) [@pitsianis](https://discourse.julialang.org/u/pitsianis)\
**Post date:** [November 8, 2024, 3:42pm UTC](https://discourse.julialang.org/t/julia-access-to-apple-gpu-with-mlx-and-or-metal-performance-shaders-mps/117647/4 "2024-11-08T15:42:47Z")

</div>

> [@stemann](#):
>
> MPS is not a PyTorch Framework.

Of course, you are right.

---

<div class="post-metadata">

**Author:** ![stemann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stemann/32/4030_2.png) [@stemann](https://discourse.julialang.org/u/stemann)\
**Post date:** [November 9, 2024, 10:57am UTC](https://discourse.julialang.org/t/julia-access-to-apple-gpu-with-mlx-and-or-metal-performance-shaders-mps/117647/5 "2024-11-09T10:57:26Z")

</div>

Took a stab at building MLX from source: [WIP Added MLX by stemann · Pull Request #9761 · JuliaPackaging/Yggdrasil · GitHub](https://github.com/JuliaPackaging/Yggdrasil/pull/9761)

(but it may need to be a re-packaging of the official Python wheels instead - as [building the metal backend](https://github.com/ml-explore/mlx/blob/v0.20.0/CMakeLists.txt#L17) likely requires [the macOS/Windows-only `metal` compiler](https://developer.apple.com/metal/tools/))

---

<div class="post-metadata">

**Author:** ![stemann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stemann/32/4030_2.png) [@stemann](https://discourse.julialang.org/u/stemann)\
**Post date:** [November 18, 2024, 8:13pm UTC](https://discourse.julialang.org/t/julia-access-to-apple-gpu-with-mlx-and-or-metal-performance-shaders-mps/117647/6 "2024-11-18T20:13:22Z")

</div>

Additional WIP:

- [Added MLX\_C (MLX C) by stemann · Pull Request #9809 · JuliaPackaging/Yggdrasil · GitHub](https://github.com/JuliaPackaging/Yggdrasil/pull/9809)
- [Added MLX C wrapper by stemann · Pull Request #2 · stemann/MLX.jl · GitHub](https://github.com/stemann/MLX.jl/pull/2)