# Extremely high memory consumption on CPU

**URL:** <https://discourse.julialang.org/t/extremely-high-memory-consumption-on-cpu/124733>\
**Category:** General Usage\
**Tags:** benchmarktools, flux\
**Created:** [January 13, 2025, 5:00pm UTC](https://discourse.julialang.org/t/extremely-high-memory-consumption-on-cpu/124733 "2025-01-13T17:00:10Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![cirobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cirobr/32/219994_2.png) [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Post date:** [January 13, 2025, 5:00pm UTC](https://discourse.julialang.org/t/extremely-high-memory-consumption-on-cpu/124733/1 "2025-01-13T17:00:10Z")

</div>

Cheers. I have developed a model for semantic segmentation in Flux. It has around 7M parameters. Have checked its inference performance for segmenting two classes, when an array with size (512,512,3,1) is at the input.

The outcome from BenchmarkTools.jl is shown at the below table for the same pc. In the first row, gpu is disabled. It really calls the attention the memory figure in the GB range for the cpu case, while the gpu case is in KB case. I wonder if BenchmarkTools is not considering GPU memory for the metric? For the CPU case, does the metric mean the model is too expensive for running on limited devices such as IoT application processors?

As a side question: its speed on gpu is 2-3X lower than equivalent models found elsewhere on GitHub. Have already applied many recommendations to reduce allocation, with little improvement. Any hint on where to look at for improvement tips is welcome.

Thanks in advance.

![image](https://global.discourse-cdn.com/julialang/original/3X/0/2/02c98d8d82ea003fec5509797454b2e17bd7e5ff.png)

---

<div class="post-metadata">

**Author:** ![raman\_kumar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raman_kumar/32/26782_2.png) [@raman\_kumar](https://discourse.julialang.org/u/raman_kumar)\
**Post date:** [January 13, 2025, 5:15pm UTC](https://discourse.julialang.org/t/extremely-high-memory-consumption-on-cpu/124733/2 "2025-01-13T17:15:07Z")

</div>

You can use @allocated macro to know allocation at each step where you have doubt. For detailed information about [profiling](https://docs.julialang.org/en/v1/manual/profile/#Profiling) you can use PProf.jl.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [January 13, 2025, 6:20pm UTC](https://discourse.julialang.org/t/extremely-high-memory-consumption-on-cpu/124733/3 "2025-01-13T18:20:22Z")

</div>

> [@cirobr](#):
>
> I wonder if BenchmarkTools is not considering GPU memory for the metric?

This is correct. To capture information about GPU allocations, check out the tools mentioned in [Benchmarking & profiling · CUDA.jl](https://cuda.juliagpu.org/stable/development/profiling/#Time-measurements).

> [@cirobr](#):
>
> For the CPU case, does the metric mean the model is too expensive for running on limited devices such as IoT application processors?

Maybe, maybe not. Equally if not more important than the total amount of memory allocated could be the maximum memory the model uses at any given point in time.

> [@cirobr](#):
>
> As a side question: its speed on gpu is 2-3X lower than equivalent models found elsewhere on GitHub. Have already applied many recommendations to reduce allocation, with little improvement. Any hint on where to look at for improvement tips is welcome.

There’s no central, definitive source, but plenty of previous discussions to search through with the obvious keywords. Just in the past little while on Discourse, I see [Unreasonable memory usage with M4 GPU](https://discourse.julialang.org/t/unreasonable-memory-usage-with-m4-gpu/123890) and [Memory usage increasing with each epoch - #14 by JoshuaBillson](https://discourse.julialang.org/t/memory-usage-increasing-with-each-epoch/121798/14). While on GitHub, [cuda gpu memory usage increasing in time · Issue #2523 · FluxML/Flux.jl · GitHub](https://github.com/FluxML/Flux.jl/issues/2523) covers much the same. There have been similar discussions on Slack.

Side note, are you aware of the existence of [Machine Learning - Julia Programming Language](https://discourse.julialang.org/c/domain/ml/24) ? Posts that are in too general a category can get lost, because some people only check specific categories frequently.