Can I manage the memory by myself?

General Usage
1

I have a code, the GC live and RSS memory increased quickly when running, while I don’t need such big memory to do basic computation. I need to allocate lots of small dict for intermidiate computing before, and the memory increase quickly; to avoid memory allocation, I pre-allocate two array to avoid repeatly allocate small dict, it put off the invreasing. But still get lots of allocation when I repeatly add some sparse array to get final one.
I am tired of avoiding allocation, and I guess memory issues I encountered is cause from GC’s lazy? (I allreadly used GC.gc() through the cycle)
So I wonder if I can pre-define some memory for long-time variables (maybe existed throughout the code, while generated gradually as the program running), and other memory for intermediate calculation, I can clear all the memory in it when I finished one cycle.
Help, thanks in advance!

2

Have a look at: GitHub - MasonProtter/Bumper.jl: Bring Your Own Stack
Could that help for your use case?

1 Like
3

Maybe this helps: GitHub - JuliaCollections/OrderedCollections.jl: Julia implementation of associative containers that preserve insertion order

It also implements LittleDict which is a ordered dictionary, that is much faster than any other AbstractDict (ordered or not) for small collections.

I don’t know if this one applies for you: GitHub - JuliaSIMD/ManualMemory.jl: Manual memory management utilities.

There is also LibC.malloc and free, but I doubt you should often use them directly. Rather with that package? And even you think it might help, see the other options.

1 Like
4

There is also StaticTools.jl.

1 Like
5

Yes. See Libc.malloc:

https://docs.julialang.org/en/v1/base/libc/#Base.Libc.malloc

and Libc.free

https://docs.julialang.org/en/v1/base/libc/#Base.Libc.free

julia> ptr = Libc.malloc(1024)
Ptr{Nothing} @0x0000000003d713c0

julia> Libc.free(ptr)

If you really want to disable GC entirely there, is

https://docs.julialang.org/en/v1/base/base/#Base.GC.enable

Provide false as an argument to turn it off. This is not recommended.

julia> GC.enable(false)
true
2 Likes
6

It’s useful, it seems only buffer the object.
But if I need to use basic operation such as insert to a array or merge two dict, it will still allocate memory from GC?

7

Currently the use of custom allocators is not supported by Julia itself (see: Provide a way to replace default malloc to investigate different allocators · Issue #35772 · JuliaLang/julia · GitHub ). You could do it in a package, but then you have to write your own package for “insert into an array” or “merge two dicts” that are for example using Bumper.jl.

1 Like
8

The overall answer to the question “Can I manage the memory by myself?” is “No”. Disabling GC is not “managing memory manually”, it’s “ignoring the reality of memory management until my program is killed by the kernel due to an Out Of Memory error”.

While there are some ways to replace some allocations with “manually managed” memory, this does not scale, at all, since close to no packages you’d actually want to use/depend on use these packages. Not to mention that they don’t play nicely together with packages that haven’t been written with differently managed memory in mind. Internal allocations in functions some packages define are not going to use the array functions provided by StaticTools.jl, they are going to use regular GC - as are things like Array or Dict you might use yourself.

4 Likes
9

So there is room for improvement. The only question from my point of view is, to which degree do we need improvements in Julia itself, and to which degrees the required improvement could be provided by packages…

1 Like
10

Julia itself doesn’t provide a generic allocation interface, and is (as far as I can tell) unlikely to get one any time soon (if at all). This would be necessary for any kind of “real” manual memory management, at least in the form of having a custom allocator (see how Rust and Zig do it).

I personally don’t really think it’s super necessary to have “true” manual memory management, at least on the level of malloc and free, because I think that that is the wrong level of abstraction to think at for modern software development (yes, this also applies to embedded…).

11

I personally don’t really think it’s super necessary to have “true” manual memory management

Well, what is ‘true’ manual memory management?

Fact is, the allocater that is used by Julia has its limitations, and being able to swap it with different implementations would be very useful in many special situations. What is the hurdle to allow this?

1 Like
12

My understanding is that there is an effort to swap in MMTk.io

https://live.juliacon.org/talk/3XBUWE

13

Well, as good as it may be to have a variety of allocators, it’s a bit removed from OP’s issue, which itself is an XY problem. It might not be helpful to introduce many experimental tools when we’re not even certain what the algorithm is allocating or what data structures it could use instead, assuming there are available libraries.

Knowing little about the algorithm, I do have a few comments.

  1. Garbage collection is probably not the issue. Given identical algorithm and inputs, manual management and GC have to do the same number of allocations and frees, just under different conditions, and manual management is not necessarily more performant. If you are triggering GC.gc() per iteration, it should have freed all unreachable memory; it is not at all lazy, which is why triggering GC that often does hurt performance compared to manual management. If your memory usage is still large or increasing after gc, then that means your live variables really are using that much memory, and you might have to figure out if you’re retaining unneeded data somewhere, a (reachable) memory leak.

  2. Mutations via long-lived variables can still cause allocations. For example, if you push! to a Vector, eventually you run out of room in its allocated buffer, and it will automatically allocate a larger buffer and copy the contents over. The variable is not reassigned, the instance is still the same one, yet an allocation can still happen. Dictionaries also have this growth feature.

  3. To limit memory usage, you need to know the upper limit of your data structures’s sizes and numbers. With base Julia this is often done by managing isbits instances on the stack or reusing a fixed number of heap-allocated mutables, e.g. mutating arrays in-place. Alternate allocators can make this easier to write, but the limits still exist there. If your algorithm really needs a large and unfixed amount of memory, no allocation scheme will change that.

1 Like
14

Well, the simple example at GitHub - MasonProtter/Bumper.jl: Bring Your Own Stack shows a speedup of a factor of three just by using a bump allocator…

Something that would also be nice to have is a hard real-time capable garbage collector which is not that hard to implement if you allow an average performance that is significantly worse compared with the default allocator…

So there are valid reasons to make it easier to integrate alternative allocators.

1 Like
15

malloc and free, as stated in the words right after your quoted part…

I’m absolutely in favor of having a more generic “allocate an object” interface, but the fact of the matter is that efforts like Bumper.jl, while neat experiments, also show the limitations: You need to use allocate and just writing [1 2 3] in some internal function cannot be captured by the package. In order to capture allocations produced by syntax like that today (and to make the management of them more “manual”, as the OP asked), we’d need an interface for swapping out the internal allocator, which we don’t currently have.