# How to remove overhead due to nested optimization with Optim package

**URL:** <https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439>\
**Category:** Optimization (Mathematical)\
**Tags:** optim\
**Created:** [July 28, 2021, 2:07pm UTC](https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439 "2021-07-28T14:07:47Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Joris\_Pinkse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joris_pinkse/32/216398_2.png) [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Post date:** [July 28, 2021, 2:07pm UTC](https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439/1 "2021-07-28T14:07:48Z")

</div>

Consider minimizing f(x, y1, y2) using a nested procedure. In an inner loop y1, y2 are minimized as a function of x and in the outer loop f( x, y1(x), y2(x) ) is minimized where y1(x), y2(x) are the solutions to the inner optimization problems.

So during each iteration of the outer loop, there is overhead in creating new optimizers for y1 and y2. How can I avoid such overhead with the Optim package?

Thanks!!

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [July 28, 2021, 3:51pm UTC](https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439/2 "2021-07-28T15:51:48Z")

</div>

The last I looked, there was no standard way to do it. I customized the Brent function to allow me to pass in the optim object and thus reuse it. I noticed very little speedup (even though almost all allocation was gone). YMMV…

---

<div class="post-metadata">

**Author:** ![Joris\_Pinkse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joris_pinkse/32/216398_2.png) [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Post date:** [July 28, 2021, 4:19pm UTC](https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439/3 "2021-07-28T16:19:31Z")

</div>

How did you reuse it? (Optim package)

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [July 28, 2021, 5:32pm UTC](https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439/4 "2021-07-28T17:32:47Z")

</div>

The structure that Optim creates and returns with optimization results and info, I just made a version of the Brent function that accepted it (modified the one from Optim). I called it directly rather than using optimize. I overwrote the fields to prevent some from incorrectly passing from one run to the next. Like I said, there is not really a standard way to do it so this way was somewhat hacked together.

---

<div class="post-metadata">

**Author:** ![Joris\_Pinkse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joris_pinkse/32/216398_2.png) [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Post date:** [July 28, 2021, 9:16pm UTC](https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439/5 "2021-07-28T21:16:33Z")

</div>

Thanks! I feared I’d have to do something like that.

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [July 28, 2021, 9:43pm UTC](https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439/6 "2021-07-28T21:43:19Z")

</div>

Can you post an example? Is the problem that the interior optimization problem is multivariate so it allocates? Also, why not just do a full optimization of all three variables at once?

---

<div class="post-metadata">

**Author:** ![Joris\_Pinkse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joris_pinkse/32/216398_2.png) [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Post date:** [July 29, 2021, 2:18pm UTC](https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439/7 "2021-07-29T14:18:54Z")

</div>

Thanks @pkofod . The problem is of the form sum\_j f\_j(x,y\_j) where there are many j’s, each f\_j is convex in y\_j but not in x. So to answer your second question, one would be converting say 100 20-dimensional convex optimization problems into one 2000-dimensional nonconvex one.

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [July 29, 2021, 7:36pm UTC](https://discourse.julialang.org/t/how-to-remove-overhead-due-to-nested-optimization-with-optim-package/65439/8 "2021-07-29T19:36:52Z")

</div>

Right. Wrt to the problem, I was asking for the example because I wanted to show you how to avoid the allocations.