In summary, you have a large (and yet sparse) optimization problem if I understand that correctly. Some one suggested Ipopt.jl.
If you use JuMP.jl you can have access to lots of solvers that are built to solve large sparse optimization problems. KNITRO.jl is my favorite, but you can indeed use Ipopt.jl via JuMP. Either JuMP or the solver (don’t recall which but I think JuMP) will automatically differentiate the problem for you so you don’t have to derive/obtain the Jacobian and Hessian yourself.