# Finding the minimum of the Bellman operator inside SDDP iterations

**URL:** <https://discourse.julialang.org/t/finding-the-minimum-of-the-bellman-operator-inside-sddp-iterations/128031>\
**Category:** Optimization (Mathematical)\
**Created:** [April 13, 2025, 9:30pm UTC](https://discourse.julialang.org/t/finding-the-minimum-of-the-bellman-operator-inside-sddp-iterations/128031 "2025-04-13T21:30:03Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mathis\_3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mathis_3/32/215843_2.png) [@Mathis\_3](https://discourse.julialang.org/u/Mathis_3)\
**Post date:** [April 13, 2025, 9:30pm UTC](https://discourse.julialang.org/t/finding-the-minimum-of-the-bellman-operator-inside-sddp-iterations/128031/1 "2025-04-13T21:30:03Z")

</div>

Hello!  
I’m currently working on a multistage stochastic optimization problem using SDDP.jl. At each iteration (and each node), I would like to solve the following problem:  
inf\_{x \in X} T(V^k)(x)  
where T is the Bellman operator and V^k is the current approximation of the value function. (Note that this is a two-stage problem)

Does anyone have suggestions on how to do this efficiently with SDDP?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 13, 2025, 9:33pm UTC](https://discourse.julialang.org/t/finding-the-minimum-of-the-bellman-operator-inside-sddp-iterations/128031/2 "2025-04-13T21:33:13Z")

</div>

Hi @Mathis_3,

Why do you want this? Are you trying to solve some sort of decision-hazard formulation? What is the full problem you are trying to solve?

---

<div class="post-metadata">

**Author:** ![Mathis\_3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mathis_3/32/215843_2.png) [@Mathis\_3](https://discourse.julialang.org/u/Mathis_3)\
**Post date:** [April 13, 2025, 9:53pm UTC](https://discourse.julialang.org/t/finding-the-minimum-of-the-bellman-operator-inside-sddp-iterations/128031/3 "2025-04-13T21:53:48Z")

</div>

Hi @odow !

I’m working on a new method to solve multistage problems with an infinite horizon. As part of this approach, I need to solve : \inf\_{x \in X} T(V^k)(x) which is a two stage problem (x represents the first stage decisions). In particular, for a given node, the problem considers the incoming state values as decision variables, and needs an outgoing state variable for each scenario.

The method I’m developing is quite general and can be applied to any problem currently supported by `SDDP.jl`.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 13, 2025, 10:04pm UTC](https://discourse.julialang.org/t/finding-the-minimum-of-the-bellman-operator-inside-sddp-iterations/128031/4 "2025-04-13T22:04:24Z")

</div>

Unfortunately this is not possible, and there are no simple work-arounds. A core design premise of SDDP.jl is that the uncertainty is realized before entering the node, and that each subproblem is a deterministic optimization problem for a single realization of the uncertainty.

> I’m working on a new method to solve multistage problems with an infinite horizon.

I guess you know this, but SDDP.jl supports discounted infinite horizons problems.

---

<div class="post-metadata">

**Author:** ![Mathis\_3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mathis_3/32/215843_2.png) [@Mathis\_3](https://discourse.julialang.org/u/Mathis_3)\
**Post date:** [April 13, 2025, 10:11pm UTC](https://discourse.julialang.org/t/finding-the-minimum-of-the-bellman-operator-inside-sddp-iterations/128031/5 "2025-04-13T22:11:59Z")

</div>

Yes, I’m aware that `SDDP.jl` supports discounted infinite-horizon problems. However, in practice, when the discount factor \beta is close to 1, convergence becomes very slow. The method I’m developing aims to address this issue.

Thanks for your response! I’ll explore modifying the structure of `SDDP.jl` slightly to see if I can implement my approach within its framework.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 13, 2025, 10:23pm UTC](https://discourse.julialang.org/t/finding-the-minimum-of-the-bellman-operator-inside-sddp-iterations/128031/6 "2025-04-13T22:23:01Z")

</div>

> when the discount factor  
> β  
> is close to 1, convergence becomes very slow

Yeah, this is mainly because of excessively long forward passes. You may want to experiment with the options in:

> <https://github.com/odow/SDDP.jl/blob/cd9d82886f07df67378e46b59f63c43a4caa900d/src/plugins/sampling_schemes.jl#L16-L49>

You could try `terminate_on_cycle = true`, or try setting some `rollout_limit`.

> I’ll explore modifying the structure of `SDDP.jl` slightly to see if I can implement my approach within its framework.

This won’t be simple. It’s probably easiest to implement a new prototype of the algorithm to prove that it works. Then if it is promising we could consider ways to integrate it into SDDP.jl.