# Avoiding Memory leaks using CuArrays

**URL:** <https://discourse.julialang.org/t/avoiding-memory-leaks-using-cuarrays/24325>\
**Category:** GPU\
**Tags:** performance, flux\
**Created:** [May 17, 2019, 1:59pm UTC](https://discourse.julialang.org/t/avoiding-memory-leaks-using-cuarrays/24325 "2019-05-17T13:59:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![HenriDeh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrideh/32/8316_2.png) [@HenriDeh](https://discourse.julialang.org/u/HenriDeh)\
**Post date:** [May 17, 2019, 1:59pm UTC](https://discourse.julialang.org/t/avoiding-memory-leaks-using-cuarrays/24325/1 "2019-05-17T13:59:35Z")

</div>

Hello,

I have read that there is no garbage collection in GPUs. I have an algorithm where I use an `Array{Array{CuArrays{Float32,1},1}}` as a buffer. From that I sample a batch that I transform in a `NTuple{CuArrays, N}` to feed a training loop in Flux with. The buffer has a fixed size and I regularly generate new elements (`Array{CuArrays{Float32,1},1}`) to replace the old ones. I think that when I do that I replace pointers to CuArrays with other ones without freeing the Vmemory. I know that a solution is to bring the CuArray back to the cpu and then erase it but I think that’s highly inefficient and very slow. My first question is: is there a way to free that memory without the costly transfer to cpu ?

Here’s an example:

```julia
buffer = Array{Array{CuArrays{Float32,1},1}}()
... populate the buffer to its fixed size.

x::Array{CuArray{Float32,1},1}()
y::Array{CuArray{Float32,1},1}()#(say that N = 2)

batch = rand(buffer, 20)
for element in batch
    push!(x, element[1])
    push!(y, element[2])
end
x = Flux.batch(x) # produces a CuArray{Float32,2}
y = Flux.batch(y) # produces a CuArray{Float32,2}
data = (x,y)
... train a network on data
newElement::Array{CuArrays{Float32,1},1} = generateanewelement()
push!(buffer, newElement)
popfirst!(buffer) #the popped element is an array of pointers, the VRAM is not freed 

```

I though that I could simply overwrite the old element with the new at the same location (which would be the most efficient way to go). Say I do that this way:

```julia
function overwriteoldelement(buffer, indexofoldest)
    buffer[indexofoldest][1] = generatenewX()
    buffer[indexofoldest][2] = generatenewY() #these two output CuArrays
end

```

I don’t think this overwrites the memory, it simply changes the pointer or something like that right ?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [May 18, 2019, 7:01am UTC](https://discourse.julialang.org/t/avoiding-memory-leaks-using-cuarrays/24325/2 "2019-05-18T07:01:40Z")

</div>

Do you observe an OOM situation? CuArray is garbage collected, so if there’s no way to access a CuArray object on the host, it’ll get collected eventually. Objects are not kept alive by having been sent to the GPU, so you don’t need to transfer back. If you want to eagerly free memory, call `CuArrays.unsafe_free!(::CuArray)`.

---

<div class="post-metadata">

**Author:** ![HenriDeh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrideh/32/8316_2.png) [@HenriDeh](https://discourse.julialang.org/u/HenriDeh)\
**Post date:** [May 19, 2019, 1:20pm UTC](https://discourse.julialang.org/t/avoiding-memory-leaks-using-cuarrays/24325/3 "2019-05-19T13:20:10Z")

</div>

Hello,

Yes I do. Try this simple code

```julia
using CuArrays, Flux
buffer = []
while true
    push!(buffer, gpu(rand(500000)))
    if length(buffer) > 500 
        popfirst!(buffer) 
     end
end

```

This quickly makes an out of memory exception. It’s clearly a memory leak since even after the julia process encounters the exception my VRAM is not freed according to the task manager.

It does not happen if I call unsafe\_free!() before popping though.

---

<div class="post-metadata">

**Author:** ![HenriDeh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrideh/32/8316_2.png) [@HenriDeh](https://discourse.julialang.org/u/HenriDeh)\
**Post date:** [May 24, 2019, 2:51pm UTC](https://discourse.julialang.org/t/avoiding-memory-leaks-using-cuarrays/24325/4 "2019-05-24T14:51:53Z")

</div>

Hi @maleadt,

During my computations I have this error message from time to time.

```julia
error in running finalizer: AssertionError(msg="Release of dead CUDAdrv.Mem.Buffer(CUDAdrv.CuPtr{Nothing}(0x000000090367fc00), 32, CUDAdrv.CuContext(Ptr{Nothing}
@0x000000002106eda0, false, true))")

```

It does not throw an exception and my computation goes on. It does not happen consistently either. I think this is related to the unsafe\_free function that I use. Do you know what this means ?