I think JuMP.jl has a nice optimal control example of this
JuMP.jl
See Example: nonlinear optimal control of a rocket · JuMP