We are running into an issue with JuMP models run in an API that are leading to our servers running out of RAM. Oddly, it does appear that the memory consumption plateaus after some time/iterations. Also, it can be recreated outside of the API. A MWE is: using REoptLite, JuMP, CPLEX for _ in rang…

I found this issue that matches the description of my issue, but it looks like a fix was merged into master by following the issue links: data race in GC alloc counters · Issue #27173 · JuliaLang/julia · GitHub make GC counters thread-local by JeffBezanson · Pull Request #32217 · JuliaLang/julia ·…

So if you set THREADS=1, all those plots with growing memory become very stable? Maybe you can send them for reference here. Or maybe is just the program that is so slow now that it has no time to reach the memory growth ramp? Any chance you can plot with a minimum number of iteration?

See this chart above in my comment with the header “3. JuMP+Xpress JULIA_NUM_THREADS=1 and Xpress THREADS=1”. Would it be informative to run more cases with THREADS=1?

[image] nlaws: Is there a way to make a MWE of calling a multithreading subprocess without JuMP+solver? I am using stress-ng and the Base.run command to recreate a single threaded Julia container calling a multithreaded subprocess. I will update here when I have some charts to share.

Adding a StackOverflow post with the same issue (unanswered): https://stackoverflow.com/questions/52380799/memory-doesnt-get-freed-in-multiple-threads

Here is another related post (unanswered): [image] How to debug slow memory leak from threads? General Usage I’m running a calculation about 10 million times with different parameter sets. The input data is 50MB and the output of all 10 million runs adds up to 1GB. When …

It is interesting that all other examples have allocation done by Julia. Then it makes sense GC no freeing stuff. What is weird about your problem is that the allocations are done by Xpress/CPLEX. Should they be freeing the memory? There is no Julia code on threads, right? perhaps callbacks? Have…

When I ran a roughly equivalent problem in Mosel (knapsack.mos above) I did not see any memory growth. We also did not have any memory growth issues when we used Python to call Mosel (via subprocess). It seems like the memory issue is related to threading and garbage collection in Julia, but I am no…

Calling GC.gc(), can lead to large slowdown indeed, that is even a warning in the docstring. Since you tried python calling mosel, you could try julia calling mosel with run. What you think? It would be nice to see this graph: [image] Memory consumption growth with many large MIL…

I’m confused as to what the actual problem is here. As far as I understand it: You want to solve multiple calls to reopt in a single (serial) Julia instance. Each solve uses Xpress, which parallelizes over Xpress.THREADS in the branch-and-bound Overtime, the memory allocated by this Julia instance…

Memory consumption growth with many large MILP's in JuMP

Specific Domains Optimization (Mathematical)

nlaws June 15, 2021, 7:20pm 36

Here is another related post (unanswered):

Topic		Replies	Views
Excessive memory usage when adding millions of variables to a JuMP Model Optimization (Mathematical) jump , gurobi	12	463	July 12, 2025
JuMP, excessive memory usage? Optimization (Mathematical) question	16	3357	February 14, 2018
Is JuMP.@variable slow when the number gets large? Optimization (Mathematical) question , jump , gurobi	12	215	April 5, 2026
Optimal model creation (and garbage collection) Optimization (Mathematical)	20	1588	December 7, 2021
Memory management in CPLEX.jl Optimization (Mathematical) question , jump , cplex	9	497	February 11, 2024

Memory consumption growth with many large MILP's in JuMP

Related topics