# Memory usage increasing with each epoch

**URL:** https://discourse.julialang.org/t/memory-usage-increasing-with-each-epoch/121798
**Category:** Machine Learning
**Tags:** cuda, flux
**Created:** [October 26, 2024, 9:45pm UTC](https://discourse.julialang.org/t/memory-usage-increasing-with-each-epoch/121798 "2024-10-26T21:45:41Z")
**Posts on this page:** 1
**Showing post:** 2

<div class="post-metadata">

### Author: ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)
#### Post date: [October 30, 2024, 9:19am UTC](https://discourse.julialang.org/t/memory-usage-increasing-with-each-epoch/121798/2 "2024-10-30T09:19:12Z")

</div>

This is a recurring problem with Flux and Julia in general. You can see similar topics here:

> [@Flux memory usage high in SRCNN](https://discourse.julialang.org/t/flux-memory-usage-high-in-srcnn/115174):
>
> Hello, First let me preface this by saying I am new to Julia and machine learning. I just wanted to dip my toes into machine learning for fun, started out with OCR using an MLP first and now I’m trying to implement a super resolution neural network (SRCNN), with a lot of help from ChatGPT, the original paper and other resources. This is all of my code, which I think is already the MWE for my problem. using Flux using Images using JLD2 # Create super resolution convolutional neural network mo…

> [@Out of memory using Flux CNN during back propagation phase](https://discourse.julialang.org/t/out-of-memory-using-flux-cnn-during-back-propagation-phase/24492):
>
> I want to train a CNN model on the German Traffic Sign Dataset by adapting one of the model\_zoo examples. First I tried training on a GTX1070 GPU with 8GB of RAM and failed with out of memory. After that I tried training on CPU (with 16GB of RAM) and also failed with out of memory.I tried using batchsizes of 64 and 16 with the same problem. Is this model too complex for my machine? The model is: model = Chain( # First convolution, operating upon a 32x32 image Conv((3, 3), 1=\>32, pa…

> [@Flux + GPU memory problems](https://discourse.julialang.org/t/flux-gpu-memory-problems/79783):
>
> I was trying to train some models available in the Metalhead (0.7) + Flux (0.13) using an NVIDIA GTX 1080 Ti (11 GB ) using some toy data (512 images size:224x224x3). I was running the code that I show at the end of the post using Metalhead.MobileNetv3(nclasses=2), Metalhead.ResNet34(pretrain=false,nclasses = 2) . However I was not able to run (memory problems) with Metalhead.ResNet50(pretrain=false,nclasses = 2) or Metalhead.ConvNeXt(:tiny, nclasses=2). \*\*With other training platforms, I was a…

> [@Memory usage increasing with each epoch](https://discourse.julialang.org/t/memory-usage-increasing-with-each-epoch/121798):
>
> I’m having a problem where memory usage is gradually increasing with each epoch when training large neural networks with Flux (v0.14.22) and CUDA (v5.5.2). At the same time, training appears to get progressively slower as the memory usage grows. When this slowdown occurs, I observe that my GPU is being used much less effectively. For example, at the start of training I will see a constant usage of around 70%, which drops to 40% after a few epochs. Upon further investigation, I found that this p…

There can be few problems.

- If you have type instability, it consumes memory.
- Sometimes, you keep julia compiling new version of the same function. This happens usually when you do something like `cat(x...)` where x is an array or tuple of different length.
- The, there might not be enough. pressure on GC. Try to run `GC.gc()` after every iteration. It will be slow, but the memory will not grow, you will see that this solves the problem.

---

_[View the full topic](https://discourse.julialang.org/t/memory-usage-increasing-with-each-epoch/121798)._