# CUDA How to limit the gpu memory available?

**URL:** <https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599>\
**Category:** GPU\
**Tags:** fftw, cuda, memory, memory-allocation, cuarrays\
**Created:** [November 21, 2022, 6:05pm UTC](https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599 "2022-11-21T18:05:04Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ludovic\_Dumoulin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ludovic_dumoulin/32/22414_2.png) [@Ludovic\_Dumoulin](https://discourse.julialang.org/u/Ludovic_Dumoulin)\
**Post date:** [November 21, 2022, 6:05pm UTC](https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599/1 "2022-11-21T18:05:04Z")

</div>

Hello,

With other lab members we share a jupyter hub server with a single gpu (A30).

The problem is that when one is using FFTW or CuArray operations he fills the entire memory.  
When one runs a code with only low level kernel (to get ride of array operations) the memory used is constant around 1 or 2Gb (which is ok because we have 23Gb).  
Unfortunately I don’t want to rewrite the FFT operator, then I was wondering if there is a way to limit the amount of memory that a simulation can use ? (knowing that the code run fine on an other gpu with only 2Gb of memory)  
I saw this

```julia
CUDA.usage_limit[] = 2^10

```

but I think it is not working anymore ? (usage\_limit not defined)

Thank you for your help,

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [November 22, 2022, 5:30pm UTC](https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599/2 "2022-11-22T17:30:54Z")

</div>

> [@Ludovic\_Dumoulin](#):
>
> but I think it is not working anymore ? (usage\_limit not defined)

NVIDIA removed this capability from their stream-ordered allocator. I’ve filed an issue, and they are considering moving it back. It might be possible to implement an alternative awaiting that, [Re-instate memory limit · Issue #1670 · JuliaGPU/CUDA.jl · GitHub](https://github.com/JuliaGPU/CUDA.jl/issues/1670).

---

<div class="post-metadata">

**Author:** ![Ludovic\_Dumoulin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ludovic_dumoulin/32/22414_2.png) [@Ludovic\_Dumoulin](https://discourse.julialang.org/u/Ludovic_Dumoulin)\
**Post date:** [November 23, 2022, 7:54am UTC](https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599/3 "2022-11-23T07:54:27Z")

</div>

Thank you for your answer!

But I have to admit that I don’t understand the alternative.  
As I am using Julia mainly for hydrodynamics/non-linear physics simulations and data analysis I am completely lost.  
Can I set a ratio of memory used/memory reserved ?  
Or can I change the CU\_MEMPOOL\_ATTR\_RELEASE\_THRESHOLD ?  
Or I have to wait for changes from Nvidia or install a previous version of CUDA ?

Thank you again:)

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [November 23, 2022, 4:13pm UTC](https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599/4 "2022-11-23T16:13:40Z")

</div>

It’s currently not supported. We could add a workaround to CUDA.jl, but that needs somebody to do the work (hence the issue I linked to). Alternatively, you could wait for NVIDIA to add the feature back.

---

<div class="post-metadata">

**Author:** ![Ludovic\_Dumoulin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ludovic_dumoulin/32/22414_2.png) [@Ludovic\_Dumoulin](https://discourse.julialang.org/u/Ludovic_Dumoulin)\
**Post date:** [November 24, 2022, 8:56am UTC](https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599/5 "2022-11-24T08:56:58Z")

</div>

I am sorry, I was not familiar (and I am still learning about) with the “collaborative development” : package, GitHub, issue, etc…  
I’ll wait for NVIDIA 🤞  
Thank you,

---

<div class="post-metadata">

**Author:** ![Ludovic\_Dumoulin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ludovic_dumoulin/32/22414_2.png) [@Ludovic\_Dumoulin](https://discourse.julialang.org/u/Ludovic_Dumoulin)\
**Post date:** [November 24, 2022, 4:42pm UTC](https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599/6 "2022-11-24T16:42:38Z")

</div>

Would it be possible to use MIG (Multi-Instance GPU) with CUDA.jl ?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [November 24, 2022, 7:51pm UTC](https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599/7 "2022-11-24T19:51:54Z")

</div>

Yes, you can just split up the device and pick one using `CUDA_VISIBLE_DEVICES`. CUDA.jl _should_ support them directly as well (i.e. using `CUDA.device!` to select them instead of limiting which ones CUDA.jl can see via `CUDA_VISIBLE_DEVICES`).

---

<div class="post-metadata">

**Author:** ![Ludovic\_Dumoulin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ludovic_dumoulin/32/22414_2.png) [@Ludovic\_Dumoulin](https://discourse.julialang.org/u/Ludovic_Dumoulin)\
**Post date:** [November 25, 2022, 9:56am UTC](https://discourse.julialang.org/t/cuda-how-to-limit-the-gpu-memory-available/90599/8 "2022-11-25T09:56:12Z")

</div>

Nice ! I’ll try to do that ! Thank you !