Dear All, sorry for bothering with a possibly stupid question. I would like to ask for some advice about solving the Hamilton-Jacobi-Bellman equation using deep learning. I am trying to solve an HJB equation of a simple neoclassical growth model from macroeconomics. Its HJB equation is \rho V(k_t…

[image] Honza9723: I would be really glad for any insight, especially from Sorry, I am not familiar with deep learning. As you mention, there aren’t natural boundary conditions for these kind of problems. In practice, I find that a good initial guess matters a lot. I usually find @article{d…

@Tamas_Papp Thank you, I will try that! So, from your experience Chebyshev/Spline collocation tend to converge to the right solution of HJB even without imposing boundary conditions?

Yes. Again, with a good initial guess. That’s something I would invest in.

@Honza9723 this looks really cool. You want to restrict the neural network to be 1: monotonically increasing 2: strictly concave I don’t know how to do this, but if someone else does I’m all ears ( @ChrisRackauckas )… More importantly, you wanna use the state boundary condition : Let k_{min} be t…

Remark in passing: if you’re solving for the value function directly (rather than the marginal value function) and if you’re going to explore the logarithmic case, you may want to change the utility function to the standard \frac{c^{1-\gamma}-1}{1-\gamma} (i.e. don’t forget the -1).

[image] Albert_Zevelev: You want to restrict the neural network to be 1: monotonically increasing 2: strictly concave I don’t know how to do this, but if someone else does I’m all ears ( @ChrisRackauckas )… I’m not sure how to do it other than imposing these charcteristics via sampling in th…

Maybe you can use the NN to represent the second derivative (then you just need to enforce it to be negative) and integrate it to get the result?

Something is not right… if V is decreasing, and the error from foc is small, consumption must be negative then. What if you change \gamma to a non-integer, say 2.1? This can force consumption to be positive (otherwise the c^{-\gamma} will yield NaN) and v to be increasing.

I don’t know too much about that paper and deep learning, but here is my general conjecture: Boundary conditions are crucial in pinning down the exact solution to the PDE. Without proper defined boundary conditions it’s not surprising that you didn’t get anything sensible. When the PDE is solved u…

Hi @RangeFu I tried to bypass this problem by imposing a state-constraint boundary condition that would pin down value function, but this boundary condition made the training process divergent (both ADAM and standard stochastic gradient descent diverged). From my experience, after starting from ran…

Solving HJB PDE using deep learning

Specific Domains Numerics

ChrisRackauckas December 18, 2021, 8:40am 13

Yeah this probably needs the adaptive loss functions that we’re working on. Can you add the not working ones to the repo? @kirillzubov @zobot it would be good to as these as testing functions.

Topic		Replies	Views
Solve a routine economics PDE from an HJB w/ ModelingToolkit Modelling & Simulations economics , pde , finance	33	3170	April 21, 2021
Solving a PDE which comes from an HJB in DifferentialEquations.jl General Usage diffeq	15	3487	September 9, 2019
Black scholes pde General Usage latex	3	1324	December 31, 2020
NeuralODE: optimal control action in a constrained optimization problem Machine Learning diffeq , flux , optimization , machine-learning , neural-network	13	2178	July 26, 2021
How do I debug this in diffeqflux? General Usage	38	2001	November 15, 2020

Solving HJB PDE using deep learning

Related topics