# A lot of memory only loading modules!
**URL:** https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295
**Category:** Performance
**Tags:** package, memory
**Created:** [September 11, 2024, 7:48pm UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295 "2024-09-11T19:48:17Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Ronneesley](https://discourse.julialang.org/u/Ronneesley)
#### Post date: [September 11, 2024, 7:48pm UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295/1 "2024-09-11T19:48:17Z")
Hi,
I’m study why my modules requires a lot of memory `> 500MB`.
So I decided to study the growth of memory of my modules.
I discovered that only one module `using LibPQ` let Julia use 213,1 MB. If I compare only Julia REPL with 84,7 MB, I can conclude that `LibPQ` use 128,4 MB, only with `using` command! Why this happens?
Julia REPL, I can understand that there is all infrastructure of “Virtual Machine” like Java, but why `LibPQ` use more memory than Julia “base”?
But it’s not exclusive of `LibPQ`! When I load `using MySQL` my memory grow to 250,2 MB of usage (165,5 MB only with MySQL)!
**Note** : I’m only `using` the module. I never call any function of module. Only the `using` command do grow of memory usage.
---
### Author:  [@nsajko](https://discourse.julialang.org/u/nsajko)
#### Post date: [September 11, 2024, 8:58pm UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295/2 "2024-09-11T20:58:55Z")
> [@Ronneesley](#):
>
> When I load `using MySQL` my memory grow to 250,2 MB of usage (165,5 MB only with MySQL)!
FTR I can’t reproduce.
Is this in the session which precompiled MySQL.jl and dependencies? If so, the memory usage is, I guess, a side-effect of the compilation itself, and, maybe (hopefully) it’s reclaimable by the OS?
Another thing to check is whether you can reproduce this with a C program, without using Julia.
After you can reliably reproduce this, try using the [heap snapshot](https://docs.julialang.org/en/v1/stdlib/Profile/#Heap-Snapshots) functionality to investigate further.
---
### Author:  [@Ronneesley](https://discourse.julialang.org/u/Ronneesley)
#### Post date: [September 11, 2024, 9:56pm UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295/3 "2024-09-11T21:56:09Z")
I @nsajko ,
It isn’t the same session. After `add MySQL`, I did `using MySQL` for compile first time. So I closed the REPL, then opened two or three times collecting the memory usage.
At this moment I’m in a computer that I almost installed MySQL and LibPQ. I had same behaviour.
Note: the last version os MySQL is throwing a warning while compiling and recompile all tries. But LibPQ is working well, although a lot of memory.
---
### Author:  [@nsajko](https://discourse.julialang.org/u/nsajko)
#### Post date: [September 11, 2024, 10:00pm UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295/4 "2024-09-11T22:00:32Z")
> [@Ronneesley](#):
>
> the last version os MySQL is throwing a warning while compiling and recompile all tries
I don’t get that with the last version (v1.4.6) of MySQL. You’re probably on an older version.
---
### Author:  [@Ronneesley](https://discourse.julialang.org/u/Ronneesley)
#### Post date: [September 12, 2024, 3:32am UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295/5 "2024-09-12T03:32:41Z")
It’s true @nsajko , I don’t now why, but a new system added old version 1.3.1 (I think). After update to MySQL v1.4.6 the module load only about 20 MB.
So the problem is some modules that I have at my systems. LibPQ for sample, still same the problem leading 212,4 MB after `using LibPQ` (about 128 MB, only at loading the module). At this case my version is LibPQ v1.18.0.
It’s so weird, when a updated MySQL with comand `] update` MySQL upgraded but LibPQ not! When I did `add LibPQ@1.18.0`, MySQL downgraded. When I run `] update` again, LibPQ downgraded and MySQL upgraded. 😕
I’ll try this at virtual enviroment and see the behavior.
At a nutshell the problem are modules, at this case `LibPQ` is bad optimizated.
---
### Author:  [@abraemer](https://discourse.julialang.org/u/abraemer)
#### Post date: [September 12, 2024, 5:56am UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295/6 "2024-09-12T05:56:16Z")
This sounds a bit like you have a very big messy environment. Post the output of `Pkg.status()` to confirm.
Try your experiments in a clean environment or a temporary one (which you get with `Pkg.activate(;temp=true)`).
---
### Author:  [@Ronneesley](https://discourse.julialang.org/u/Ronneesley)
#### Post date: [September 13, 2024, 12:37pm UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295/7 "2024-09-13T12:37:48Z")
Let me see the messy @abraemer ,
Case 1:
```julia
(@v1.10) pkg> activate env-a
Activating new project at `C:\Users\ronne\env-a`
(env-a) pkg> add MySQL LibPQ
(env-a) pkg> status
Status `C:\Users\ronne\env-a\Project.toml`
⌃ [194296ae] LibPQ v1.17.1
[39abe10b] MySQL v1.4.6
```
Case 2:
```julia
(env-a) pkg> activate env-b
Activating new project at `C:\Users\ronne\env-b`
(env-b) pkg> add LibPQ
(env-b) pkg> status
Status `C:\Users\ronne\env-b\Project.toml`
[194296ae] LibPQ v1.18.0
```
Case 3:
```julia
(env-b) pkg> activate env-c
Activating new project at `C:\Users\ronne\env-c`
(env-c) pkg> add MySQL
(env-c) pkg> status
Status `C:\Users\ronne\env-c\Project.toml`
[39abe10b] MySQL v1.4.6
```
So, certainly there is some kind of messy because the last version of these two packages `MySQL` and `LibPQ` are imcompactibles. Why when I installed the two packages, `LibPQ` is getting version 1.17.1 although there is version 1.18.0?
This way, there really is a very big messy, @abraemer . A messy not only at my enviroment, but at this two packages.
---
### Author:  [@abraemer](https://discourse.julialang.org/u/abraemer)
#### Post date: [September 13, 2024, 1:43pm UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295/8 "2024-09-13T13:43:15Z")
Sorry, I didn’t mean to offend you. A weird upgrading/downgrading of packages is just a common consequence of large environments (possibly containing obsolete packages).
I had a brief look and found:
```julia-repl
julia> Pkg.activate(;temp=true)
(jl_zwHUIh) pkg> add LibPQ MySQL
# ...
(jl_zwHUIh) pkg> st
Status `/tmp/jl_zwHUIh/Project.toml`
⌃ [194296ae] LibPQ v1.17.1
[39abe10b] MySQL v1.4.6
(jl_zwHUIh) pkg> status --outdated -m
Status `/tmp/jl_zwHUIh/Manifest.toml`
⌅ [fa961155] CEnum v0.4.2 ( why OpenSSL_jll
LibPQ → LibPQ_jll → OpenSSL_jll
MySQL → MariaDB_Connector_C_jll → OpenSSL_jll
```
which tells me that the problem is likely `OpenSSL_jll`.
Trying to add the latest versions of both gives more info:
```julia-repl
(jl_zwHUIh) pkg> add MySQL@1.4.6 LibPQ@1.18.0
Resolving package versions...
ERROR: Unsatisfiable requirements detected for package MariaDB_Connector_C_jll [aabc7e14]:
MariaDB_Connector_C_jll [aabc7e14] log:
├─possible versions are: 3.1.6-3.3.9 or uninstalled
├─restricted by compatibility requirements with MySQL [39abe10b] to versions: 3.1.12
│ └─MySQL [39abe10b] log:
│ ├─possible versions are: 0.6.0-1.4.6 or uninstalled
│ └─restricted to versions 1.4.6 by an explicit requirement, leaving only versions: 1.4.6
└─restricted by compatibility requirements with OpenSSL_jll [458c3c95] to versions: 3.3.9 or uninstalled — no versions left
└─OpenSSL_jll [458c3c95] log:
├─possible versions are: 1.1.1-3.0.15 or uninstalled
└─restricted by compatibility requirements with LibPQ_jll [08be9ffa] to versions: 3.0.8-3.0.15
└─LibPQ_jll [08be9ffa] log:
├─possible versions are: 12.3.0-16.0.0 or uninstalled
└─restricted by compatibility requirements with LibPQ [194296ae] to versions: 16.0.0
└─LibPQ [194296ae] log:
├─possible versions are: 0.4.0-1.18.0 or uninstalled
└─restricted to versions 1.18.0 by an explicit requirement, leaving only versions: 1.18.0
```
I can follow the compatibilities somewhat:
- [LibPQ 1.18.0 requires](https://github.com/JuliaDatabases/LibPQ.jl/blob/ae7e231fe9f2db88cf731de84a471fac8e168436/Project.toml#L36) LibPQ\_jll 16.
- [LibPQ\_jll requires at least](https://github.com/JuliaPackaging/Yggdrasil/blob/069e6dea171da76990076578e469e39f10d3228c/L/LibPQ/build_tarballs.jl#L90) OpenSSL\_jll 3.0.8
- [MySQL 1.4.6 requires exactly](https://github.com/JuliaDatabases/MySQL.jl/blob/9d61afbc137284b830b259a873f50c028c30615b/Project.toml#L18) MariaDB\_Connector\_C\_jll 3.1.12
- But MariaDB\_Connector\_C\_jll 3.1.12 [seems to not constrain](https://github.com/JuliaPackaging/Yggdrasil/blob/f16b91e37fa7043a0847177e88a2bde10545d31f/M/MariaDB_Connector_C/build_tarballs.jl#L75) OpenSSL\_jll IIUC
So I don’t really understand how this happens… Perhaps @mkitti or @kristoffer.carlsson do know more?
Edit: I just saw that the [Project.toml in JuliaBinaryWrappers/MariaDB\_Connector\_C\_jll repository](https://github.com/JuliaBinaryWrappers/MariaDB_Connector_C_jll.jl/blob/MariaDB_Connector_C-v3.1.12%2B0/Project.toml) for tag 3.1.12 contains the version 3.1.6. I wonder whether this is the root of this issue.
---
### Author:  [@danielwe](https://discourse.julialang.org/u/danielwe)
#### Post date: [September 14, 2024, 4:35am UTC](https://discourse.julialang.org/t/a-lot-of-memory-only-loading-modules/119295/9 "2024-09-14T04:35:03Z")
> [@abraemer](#):
>
> - But MariaDB\_Connector\_C\_jll 3.1.12 [seems to not constrain](https://github.com/JuliaPackaging/Yggdrasil/blob/f16b91e37fa7043a0847177e88a2bde10545d31f/M/MariaDB_Connector_C/build_tarballs.jl#L75) OpenSSL\_jll IIUC
The bound was inserted by hand directly in the registry rather than through Project.toml, see [Set compat for OpenSSL\_jll to 1.1.10-1 for OpenSSL\_jll v3 transition by mkitti · Pull Request #78542 · JuliaRegistries/General · GitHub](https://github.com/JuliaRegistries/General/pull/78542/files#diff-f8fa8ef463afd893110eda6cda5d54040964db182d68d5a7c5ccd7f5cfad27bd), and then relaxed again for But MariaDB\_Connector\_C\_jll \> 3.3.2 in [New version: MariaDB\_Connector\_C\_jll v3.3.9+0 by jlbuild · Pull Request #104208 · JuliaRegistries/General · GitHub](https://github.com/JuliaRegistries/General/pull/104208/files#diff-222e89504ab704780bf8f17e185df17f29ae698f6a54b57c4b520e9839d3437f)