# Best practice for large-scale models in JuMP?

**URL:** <https://discourse.julialang.org/t/best-practice-for-large-scale-models-in-jump/37256>\
**Category:** Optimization (Mathematical)\
**Tags:** question, jump\
**Created:** [April 8, 2020, 11:25pm UTC](https://discourse.julialang.org/t/best-practice-for-large-scale-models-in-jump/37256 "2020-04-08T23:25:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![olugovoy](https://avatars.discourse-cdn.com/v4/letter/o/a698b9/32.png) [@olugovoy](https://discourse.julialang.org/u/olugovoy)\
**Post date:** [April 8, 2020, 11:25pm UTC](https://discourse.julialang.org/t/best-practice-for-large-scale-models-in-jump/37256/1 "2020-04-08T23:25:13Z")

</div>

Hi there,

We are translating a large scale LP model from GAMS to Julia/JuMP.  
Large-scale means, there are a number of sets (regions, years, sub-annual time resolution - days and hours, technologies, etc.) and 2-5 dimensional variables, and the matrix (before aggregation by CPLEX) might be 5e+7 by 5e+7 with 3e+7 non-zeros. It is solvable in GAMS and Pyomo. Would be great to have JuMP version.

Here is an example of a small model which works well: [UTOPIA\_BASE\_JuMP.7z](https://www.dropbox.com/transfer/AAAAACWnUqUoEwc0ez6TTp5ZGV71997Nhw4-QHwmvLZ02iG6Fc9LbYA)

But we have encountered a limitation of this approach. It doesn’t work for large-scale models.

First, it seems to be not the best from performance perspective.

Second, there is some limitation in length of dictionaries (see my other question: [#37247](https://discourse.julialang.org/t/a-way-to-create-large-dictionary-error-loaderror-syntax-expression-too-large/37247))

The sets in our model are character vectors (“Symbols” & “Dictionaries” in Julia). And the preference would be keep it this way, using names of regions etc. rather than numbered sets. Though performance is the priority. Would be sparse arrays a better alternative to Dicts/Tuples?

Any thoughts on how to define large-scale models and improve time-performance will be appreciated.

---

<div class="post-metadata">

**Author:** ![ccoffrin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ccoffrin/32/400_2.png) [@ccoffrin](https://discourse.julialang.org/u/ccoffrin)\
**Post date:** [April 9, 2020, 1:07am UTC](https://discourse.julialang.org/t/best-practice-for-large-scale-models-in-jump/37256/2 "2020-04-09T01:07:11Z")

</div>

I don’t think you will have any issue using “Symbols” & “Dictionaries” in Julia and in JuMP. I recomend starting with resolving #37247, this would be a Julia-only issue to resolve. Once that is working, then I would work on the scalability of the JuMP model.

I personally have never run into any scalability issues with JuMP. So I think it is worth the effort to work through any issues you might be experiencing.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 9, 2020, 2:23am UTC](https://discourse.julialang.org/t/best-practice-for-large-scale-models-in-jump/37256/3 "2020-04-09T02:23:12Z")

</div>

First, read the performance tips: [Performance Tips · The Julia Language](https://docs.julialang.org/en/v1/manual/performance-tips/index.html)

In particular, don’t use global variables. This can largely be achieved by 1) putting you code in a function or 2) prepending `const =` to you lines in `data.jl`.

You should also benchmark your code to identify where the slow points are.

If you data is sparse, JuMP uses a standard dictionary as the backing data structure anyway.

In order to get specific feedback, you should distill a small, minimal working example that demonstrates your code structure.

As one simplification:

```nohighlight
if haskey(foo, key)
    foo[key]
else
   bar
end

# use

get(foo, key, bar)

```