# Out of memory using Flux CNN during back propagation phase

**URL:** https://discourse.julialang.org/t/out-of-memory-using-flux-cnn-during-back-propagation-phase/24492
**Category:** Machine Learning
**Created:** [May 22, 2019, 7:56pm UTC](https://discourse.julialang.org/t/out-of-memory-using-flux-cnn-during-back-propagation-phase/24492 "2019-05-22T19:56:49Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Iulian.Cioarca](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iulian.cioarca/32/30166_2.png) [@Iulian.Cioarca](https://discourse.julialang.org/u/Iulian.Cioarca)
#### Post date: [May 22, 2019, 7:56pm UTC](https://discourse.julialang.org/t/out-of-memory-using-flux-cnn-during-back-propagation-phase/24492/1 "2019-05-22T19:56:49Z")

</div>

I want to train a CNN model on the German Traffic Sign Dataset by adapting one of the model\_zoo examples. First I tried training on a GTX1070 GPU with 8GB of RAM and failed with out of memory. After that I tried training on CPU (with 16GB of RAM) and also failed with out of memory.I tried using batchsizes of 64 and 16 with the same problem.  
Is this model too complex for my machine?  
The model is:

```julia
 model = Chain(
     # First convolution, operating upon a 32x32 image
     Conv((3, 3), 1=>32, pad=(1,1), relu),
	 Conv((3, 3), 32=>32, pad=(1,1), relu),
     BatchNorm(32),
     x -> maxpool(x, (2,2)),

     # Second convolution, operating upon a 16x16 image
     Conv((3, 3), 32=>64, pad=(1,1), relu),
	 Conv((3, 3), 64=>64, pad=(1,1), relu),
     BatchNorm(64),
     x -> maxpool(x, (2,2)),

     # Third convolution, operating upon a 8x8 image
     Conv((3, 3), 64=>128, pad=(1,1), relu),
	 Conv((3, 3), 128=>128, pad=(1,1), relu),
     BatchNorm(128),
     x -> maxpool(x, (2,2)),

     x -> reshape(x, :, size(x, 4)),
	 Dense(2048, 128),
	 Dense(128, 128),
     Dense(128, 43),
     # Finally, softmax to get nice probabilities
     softmax,
 )

```

---

<div class="post-metadata">

### Author: ![jchen975](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jchen975/32/8994_2.png) [@jchen975](https://discourse.julialang.org/u/jchen975)
#### Post date: [June 22, 2019, 2:10am UTC](https://discourse.julialang.org/t/out-of-memory-using-flux-cnn-during-back-propagation-phase/24492/2 "2019-06-22T02:10:18Z")

</div>

I’m also having `OutOfMemoryError()` problems with a large batch size but I think your model isn’t that complex to be continuously causing that problem. Do you have any updates? If you’re using `CuArrays` I’ve read that you could force freeing the GPU memory in between the batches/epochs with `finalize`, which might help.

---

<div class="post-metadata">

### Author: ![Iulian.Cioarca](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iulian.cioarca/32/30166_2.png) [@Iulian.Cioarca](https://discourse.julialang.org/u/Iulian.Cioarca)
#### Post date: [June 28, 2019, 12:27pm UTC](https://discourse.julialang.org/t/out-of-memory-using-flux-cnn-during-back-propagation-phase/24492/3 "2019-06-28T12:27:14Z")

</div>

I have not tested this workaround yet because meanwhile I switched to Knet. I might give it another try in the future, but for the moment Knet does the job really well.