Solving dynamic optimization problems is at the heart of economics (and many other fields).

The goal of this post is to explore various tools for solving dynamic optimization problems in the Julia Eco-system and to hopefully see if there are opportunities for synergy between the many different communities that solve these types of problems.

This post will consider a firm that chooses capital investment to maximize its value.

Here are the 4 quadrants of modeling:

We will study the case F(K)=zK, p_t =1, and C(I,K)=I + \frac{1}{2} I^2 because it has simple closed form solutions (for comparison).

Here is the corresponding closed form solution:

More detail for the deterministic cases.

Once a user (economist/engineer/etc) chooses a modeling framework (e.g. deterministic/discrete-time) there are often many different ways to solve the same problem.

Hence, a modeler has 3 steps

- Decide how to formulate the problem (e.g. Deterministic/Continuous-Time)
- Decide which mathematical problem solve:

-Sometimes itās easier to solve a system of three BV-DAEs

-Sometimes itās easier to solve a system of two first-order BV-ODEs

-Sometimes itās easier to solve one second-order BV-ODE

-Sometimes itās easier to solve the HJB-pde (in this case itās an ODE)

-Note: not all approaches involve solving equations (e.g. see InfiniteOpt.jl) - Decide which method(s) to use to solve the mathematical problem specified above

-E.g. there are many methods a modeler can use to solve a system of two first-order difference equations (perturbation, projection, and hybrids).

There are ENORMOUS trade-offs users need to make.

For example, perturbation methods (SolveDSGE.jl) are a powerful approach to solve large systems quickly, but generally work for models w/ well defined steady-states.

They also need to be extended to solve models w/ occasionally binding constraints (OccBin is not yet implemented in SolveDSGE.jl, see Julia examples here).

Dynamic programming approaches (such as VFI) can solve a very wide variety of problems, but they tend to be very-very slow. Currently, Julia does not have a generic equivalent to Matlabās VFIToolkit. There was a Julia GSoC project that solved the optimal growth model w/ parallel VFI, but it was not generic.

**Quadrant: Discrete-time/deterministic**

Package/Sample Code | Description | Note |
---|---|---|

SolveDSGE.jl -Code link | Solves difference equations w/ Perturbation/Projection | Only for discrete time models w/ well defined steady-states @RJDennis |

POMDPs.jl -Code link | Solves Bellman equations w/ various methods | discussion @zsunberg |

JuMP.jl SDDP.jl | Description | Not sure it solves infinite horizon models. @ odow & co |

Package | Description | Note |

**Quadrant: Discrete-time/Stochastic**

Package/Sample Code | Description | Note |
---|---|---|

SolveDSGE.jl -Code link | Solves stochastic difference equations w/ Perturbation/Projection | Only for discrete time models w/ well defined steady-states @RJDennis |

POMDPs.jl | Solves Bellman equations w/ various methods | Please post code below using e.g. a 2-state Markov Chain for z. @zsunberg |

Package | Description | Note |

**Quadrant: Continuous -time/deterministic**

Package/Sample Code | Description | Note |
---|---|---|

-Code link | Solves HJB equations using Ben Mollās method | My custom solver. Works pretty nicely. |

EconPDEs.jl -Code link | Solves HJB equations | @matthieu |

InfiniteOpt.jl | Solves Hamiltonian | Only finite horizon currently. @pulsipher |

SciML.jl | Solve DEs | TO DO. Only finite horizon currently. Try @MainBatDealer & @ptoche trick to solve a DE w/ an infinite horizon boundary constraint. @ChrisRackauckas |

Flux.jl -Code link | Solve HJB w/ NN | @Goosee |

Package | Description | Note |

**Quadrant: Continuous -time/Stochastic**

Package/Sample Code | Description | Note |
---|---|---|

EconPDEs.jl | Solves HJB equations | To Do. @matthieu |

Flux.jl | Solves HJB equations w/ Deep Learning | @Goosee |

Package | Description | Note |

There are many other dynamic programming methods (VFI/PFI/ECMā¦) that currently donāt have generic/robust implementations in Julia.

@sglyon has 7+ methods

@floswald has others

Unfortunately @sglyonās code is not generic like VFIToolkit.m

To Do:

check out QuantEcon.jl

solve w/ SDDP.jl|

solve 2-state Markov Chain version w/ POMDPs.jl

solve CTS-time w/ SciML.jl

solve stochastic w/ EconPDEs.jl

solve infinite horizon w/ InfiniteOpt.jl

check out Dynare.jl (not maintained?)

Dolo.jl

DifferentiableStateSpaceModels.jl

MFGNet.jl

Oren Levintal has a state-of-the-art global package in Matlab

Robert Kirkby has a phenomenal VFI package in Matlab

Please let me know what I missed!