Dear all,
in the context of stochastic control, I am wondering if some of you have implemented either of the two papers in julia.
- Onken, D., Nurbekyan, L., Li, X., Fung, S. W., Osher, S., & Ruthotto, L. (2023). A Neural Network Approach for High-Dimensional Optimal Control Applied to Multiagent Path Finding. IEEE Transactions on Control Systems Technology, 31(1), 235–251. A Neural Network Approach for High-Dimensional Optimal Control Applied to Multiagent Path Finding | IEEE Journals & Magazine | IEEE Xplore
- Li, X., Verma, D., & Ruthotto, L. (2024). A Neural Network Approach for Stochastic Optimal Control. SIAM Journal on Scientific Computing, 46(5), C535–C556. https://doi.org/10.1137/23M155832X
Thank you for your help,
Best regards
Are those cases not already handled by HighDimPDE.jl? If not, they are algorithms of that type and should just be minor tweaks.
I would say no, “there is no control” in this package.
What do you mean? The HJB solution is the optimal controller. Here’s the example of doing those papers:
That is solving the Linear Quadratic Gaussian (LQG) control problem of 100 dimensions
I meant that you have to specify the HJB rather than the cost function. OK that’s an API requirement.
For example:
- where do you specifiy the running cost
- how do you get the optimal strategy (ie the optimal control)
- the above papers are younger than the one of Han, A. Jentzen and Weinan E. They implement a different method.
I’ll get those methods into HighDimPDE.jl. It’s actually a pretty trivial extension, since we don’t actually implement it via the Han style, their paper can be generalized as just solving a stochastic UDE, and similarly these two papers are also just a stochastic UDE at the end of the day. So solving them is just defining the SDE (which are all very similar) and then just running the standard optimization loop over it. I’m out of Claude hours but next week I’ll just set it on it and it’ll most likely just one shot it given the infrastructure that already exists.
But yes, all of these assume you write it in the HJB form. It would be nice to provide a different interface so that it’s written in the control form, and then auto-translates to the HJB, and then solves it like that. That probably would be done via a ModelingToolkit representation on top… we don’t have that piece all together but it would be a very cool addition to do.