# Is Sys.free\_memory() "reliable"?

**URL:** <https://discourse.julialang.org/t/is-sys-free-memory-reliable/116159>\
**Category:** General Usage\
**Created:** [June 24, 2024, 8:06pm UTC](https://discourse.julialang.org/t/is-sys-free-memory-reliable/116159 "2024-06-24T20:06:17Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 24, 2024, 8:06pm UTC](https://discourse.julialang.org/t/is-sys-free-memory-reliable/116159/1 "2024-06-24T20:06:17Z")

</div>

I’m trying to add a custom error message for a package if the user inputs data such that the memory of the system will explode. For such, I use:

```julia
if required_memory > 0.8 * Sys.free_memory()
    throw(.... appropriate error message... )
end

```

The problem is that after adding this to the package, CI tests started to fail, because in MacOS with Julia 1.9, it seems that `free_memory()` is returning way too small values. For instance, the error message says something like `free_memory` being 0.1Mb.

Those tests are small and do not fail normally, so it is not the case that that is the true free memory of the node running the CI test.

Are there any known issues associated to `Sys.free_memory()`, particularly in MacOS?

Ps: The error occurred in Julia 1.9 but not in 1.10.4, thus it might be a fixed bug somewhere in between.

Anyway I would like to know if that use of `free_memory` is reasonable one (I don’t want my users getting Killed for no apparent reason).

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 24, 2024, 8:14pm UTC](https://discourse.julialang.org/t/is-sys-free-memory-reliable/116159/2 "2024-06-24T20:14:56Z")

</div>

I re-run the CI job that failed, and now it passed. I’m not sure if I’m happy or not.

(I wouldn’t like the users getting the error message randomly either…)

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 24, 2024, 8:31pm UTC](https://discourse.julialang.org/t/is-sys-free-memory-reliable/116159/3 "2024-06-24T20:31:35Z")

</div>

I’m definitely not an expert on this, but I think there’s a difference between “free memory” and “available memory”, where the latter includes OS caches which can be quickly freed. I think “free memory” is kinda wasted (since it could be caching something, as long as the OS can quickly dump the cache when the memory is needed for something else), and MacOS tends to not have much of it. And Julia doesn’t seem to expose a `Sys.available_memory()` or similar.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 24, 2024, 10:15pm UTC](https://discourse.julialang.org/t/is-sys-free-memory-reliable/116159/4 "2024-06-24T22:15:28Z")

</div>

If Mac owners can confirm that `Sys.free_memory()` is generally small and not ver related to available memory, I’d be grateful.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 24, 2024, 10:34pm UTC](https://discourse.julialang.org/t/is-sys-free-memory-reliable/116159/5 "2024-06-24T22:34:23Z")

</div>

For me, I am on MacOS and get

```julia
julia> Base.format_bytes(Sys.free_memory())
"68.375 MiB"

```

while activity monitor says “memory used” is 2 gb less than “physical memory”:

 ![Screenshot 2024-06-25 at 00.30.15](https://global.discourse-cdn.com/julialang/original/3X/0/d/0db33d4a534acc58072c7461c1dd2b2ca6e0d15a.png)

so I guess I am using a large % of my memory but have available more than the tiny bit `Sys.free_memory()` says. I for one would be disappointed if a Julia package refused to allocate more than that to me; I can allocate it without an issue or noticeable slowdown:

```julia
julia> Base.format_bytes(Base.summarysize(rand(10_000_000)))
"76.294 MiB"

```

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [June 25, 2024, 7:15am UTC](https://discourse.julialang.org/t/is-sys-free-memory-reliable/116159/6 "2024-06-25T07:15:53Z")

</div>

I am on Linux (Ubuntu) and for me `Sys.free_memory` seems to count memory used for cache as “free”:

```julia-repl
julia> Sys.free_memory() |> Base.format_bytes
"10.072 GiB"

```

`htop` output (green - used, orange - cache)

 ![image](https://global.discourse-cdn.com/julialang/original/3X/0/4/0457c4d6683f73411669b42a1c7c7289840fe9e1.png)

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 25, 2024, 11:58am UTC](https://discourse.julialang.org/t/is-sys-free-memory-reliable/116159/7 "2024-06-25T11:58:14Z")

</div>

Thanks for the feedback. I decided then to issue a warning if the memory exceeds a fraction of the total memory, and only error if the required memory is larger than the total memory.