# Large scale model using Gurobi causes machine crushed

**URL:** <https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988>\
**Category:** Optimization (Mathematical)\
**Tags:** jump, gurobi\
**Created:** [October 30, 2024, 11:56am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988 "2024-10-30T11:56:17Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Betristor](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/betristor/32/45264_2.png) [@Betristor](https://discourse.julialang.org/u/Betristor)\
**Post date:** [October 30, 2024, 11:56am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/1 "2024-10-30T11:56:17Z")

</div>

For the record, I still have not found the reason why my server crushed solving some large models using JuMP+Gurobi. But I have experienced like dozens of times.  
Here is the situation. Each time I constructed a JuMP model with about 200B variables with like three instances (each takes about 250 GB using `direct mode`, the total memory consumption is within the machine limit 1.5TB), the machine will crush and no traces are left in the logs from the system and gurobi. It’s just like a random thing that I could not control at all, and I don’t know when it will happen, how it happened, and how to fix this. I had to reboot the machine since no ssh and cable connection is available when this happens.  
It’s killing me! Is there anybody who has encountered the same issue with me?

---

<div class="post-metadata">

**Author:** ![torressa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torressa/32/202204_2.png) [@torressa](https://discourse.julialang.org/u/torressa)\
**Post date:** [October 30, 2024, 5:45pm UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/2 "2024-10-30T17:45:07Z")

</div>

I encountered similar issues a while ago but it was not related to either Gurobi/JuMP and was likely a Julia issue (I was able to build and solve the model effectively once I migrated to another language).  
I’m guessing this is during the model-building phase. Have you tested other solver interfaces?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 30, 2024, 8:26pm UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/3 "2024-10-30T20:26:58Z")

</div>

Gurobi.jl wraps the 32-bit Gurobi API, so you cannot have more than `2_147_483_647` variables.

---

<div class="post-metadata">

**Author:** ![Betristor](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/betristor/32/45264_2.png) [@Betristor](https://discourse.julialang.org/u/Betristor)\
**Post date:** [October 31, 2024, 2:02am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/4 "2024-10-31T02:02:39Z")

</div>

Normally the model building is smooth, the machine crushed during the solving process.

---

<div class="post-metadata">

**Author:** ![Betristor](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/betristor/32/45264_2.png) [@Betristor](https://discourse.julialang.org/u/Betristor)\
**Post date:** [October 31, 2024, 2:10am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/5 "2024-10-31T02:10:01Z")

</div>

@odow Is there any future plan that Gurobi.jl will extend this limit? Since gurobi didn’t limit the variables and supports 64-bit. Another question is does this mean the underlying float precision is restricted to 32-bit or something else?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 31, 2024, 2:22am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/6 "2024-10-31T02:22:52Z")

</div>

There are no plans.

It doesnt make sense to try to solve a problem with that many variables. Even if you could build it, Gurobi is unlikely to solve it.

What is your application? How can you interpret a problem with 10^11 decision variables.

---

<div class="post-metadata">

**Author:** ![Betristor](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/betristor/32/45264_2.png) [@Betristor](https://discourse.julialang.org/u/Betristor)\
**Post date:** [October 31, 2024, 2:29am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/7 "2024-10-31T02:29:07Z")

</div>

@odow I constructed a problem with 8760 time slots and over 1000 locations. The high spatiotemporal resolution results in a very large variable space which is necessary for me to find the temporal dynamics and precise spatial location.  
Now since gurobi supports 2147483647 variables at most, I think I do need to reduce the resolution.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 31, 2024, 2:33am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/8 "2024-10-31T02:33:08Z")

</div>

So you have 8.76e6 time/location slots. But that doesn’t explain how you the have an additional 1e5 variables for each of the time/location pairs?

---

<div class="post-metadata">

**Author:** ![Betristor](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/betristor/32/45264_2.png) [@Betristor](https://discourse.julialang.org/u/Betristor)\
**Post date:** [October 31, 2024, 2:35am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/9 "2024-10-31T02:35:52Z")

</div>

1e11 is now in hypothesis. The most common situation is about 1.9e8 variables.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 31, 2024, 2:37am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/10 "2024-10-31T02:37:34Z")

</div>

To clarify the future plans: there are `GRBX` routines which allow more than 2e9 non-zeros in the constraint matrix: [GRBXloadmodel - Gurobi Optimization](https://www.gurobi.com/documentation/current/refman/c_xloadmodel.html). But they still don’t support more than `typemax(Cint)` variables or constraints.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 31, 2024, 2:39am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/11 "2024-10-31T02:39:45Z")

</div>

In that case you’re likely hitting the limit of the number of non-zeros in the constraint matrix.

---

<div class="post-metadata">

**Author:** ![Betristor](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/betristor/32/45264_2.png) [@Betristor](https://discourse.julialang.org/u/Betristor)\
**Post date:** [October 31, 2024, 2:40am UTC](https://discourse.julialang.org/t/large-scale-model-using-gurobi-causes-machine-crushed/121988/12 "2024-10-31T02:40:50Z")

</div>

Thanks for the replying. This is of great help. Now I finally know where the boundary is.