# Running OOM trying to load data to GPU

**URL:** <https://discourse.julialang.org/t/running-oom-trying-to-load-data-to-gpu/100480>\
**Category:** Machine Learning\
**Tags:** gpu, flux\
**Created:** [June 17, 2023, 2:01am UTC](https://discourse.julialang.org/t/running-oom-trying-to-load-data-to-gpu/100480 "2023-06-17T02:01:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![lepton01](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lepton01/32/48824_2.png) [@lepton01](https://discourse.julialang.org/u/lepton01)\
**Post date:** [June 17, 2023, 2:01am UTC](https://discourse.julialang.org/t/running-oom-trying-to-load-data-to-gpu/100480/1 "2023-06-17T02:01:46Z")

</div>

Hello there.  
After training the CNN, I wrote a function to estimate the accuracy of it.

```julia
function accuracy(A, B, name)
    BSON.@load name * ".bson" model
    model = model |> gpu
    X1, Y1 = A
    X2, Y2 = B
    Y_tr_r = model(X1 |> gpu) |> cpu
    Y_te_r = model(X2 |> gpu) |> cpu
    a = mean(isapprox.(Y_tr_r, Y1; atol=0.015)) * 100
    b = mean(isapprox.(Y_te_r, Y2; atol=0.015)) * 100
    return a, b
end

```

Problem is: the GPU runs OOM when trying to load the data to it. I do not understand why, the data is not big in size:  
X1 is `Array{Float64,4} dims=(128,128,1,5000)`, X2 is similarly `Array{Float64,4} dims=(128,128,1,1250)`, Y1 and Y2 are even smaller.  
The exact line it errors is `Y_tr_r = model(X1 |> gpu) |> cpu`  
GPU: Nvidia GeForce GTX 1660 SUPER (6 GB VRAM).  
Yes, I CUDA.reclaim() finishing training, so the VRA;M is mostly free…  
Using the CPU works, but I would like to know if using the GPU is faster to evaluate.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [June 17, 2023, 2:38pm UTC](https://discourse.julialang.org/t/running-oom-trying-to-load-data-to-gpu/100480/2 "2023-06-17T14:38:49Z")

</div>

What is `model`? If your model is large enough, it’s possible the allocations from the forward pass would be enough to OOM. Even for something the size of a Resnet-18, 128^2 with a batch size of 5000 could OOM a 8GB GPU, let alone a 6GB one (consider that batch sizes are usually \< 512)!

---

<div class="post-metadata">

**Author:** ![lepton01](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lepton01/32/48824_2.png) [@lepton01](https://discourse.julialang.org/u/lepton01)\
**Post date:** [July 8, 2023, 6:26am UTC](https://discourse.julialang.org/t/running-oom-trying-to-load-data-to-gpu/100480/3 "2023-07-08T06:26:11Z")

</div>

My apologies for not answering earlier. I do believe that was the problem, I was abusing mu GPU with ridiculous amounts of neurons after the Conv layers…