It is a pleasure to announce the release of CTBenchmarks.jl (v0.4.1), an under construction benchmarking suite that systematically compares solver–modeler combinations for optimal control problems.
Most of the optimal control problems benchmarked come from known problem sets: check references inside each problem documentation.
What is benchmarked
We conducted performance comparisons across multiple dimensions:
- Solvers: Ipopt and MadNLP (upcoming: MadNCL, UNO)
- Modelers: ADNLPModels, ExaModels, and JuMP
- Architectures: CPU (MUMPS) vs GPU (CUDSS) with ExaModels
- Problem suite: All optimal control problems from OptimalControlProblems.jl
- Grid sizes: Multiple discretization levels
- Discretization methods: Trapeze and midpoint schemes
Performance Profiles
We provide Dolan–Moré performance profiles that compare all solver–model combinations across the complete problem suite:
These profiles show the relative performance of each combination, making it easy to identify which approaches are most efficient for optimal control problems.
Problem-by-Problem Analysis
For each problem, we provide detailed comparisons showing how solve time scales with grid size:
The documentation includes:
Pre-computed benchmark results from CI runs on CPU and GPU platforms
Performance profiles for both time and iteration counts
Problem-by-problem comparisons with detailed metrics
Complete reproducibility information: Julia version, dependencies, hardware specifications
Acknowledgments
We would like to thank all contributors for their work on this package!
Contributing
This is ongoing work and contributions are welcome. Whether you want to add support for new solvers, contribute new optimal control problems, or improve the benchmarking infrastructure, feel free to open an issue or start a discussion on GitHub.
About control-toolbox
CTBenchmarks.jl is part of the control-toolbox ecosystem, which brings together Julia packages for mathematical control and its applications:
- OptimalControl.jl - Tools to model and solve optimal control problems with a friendly DSL
- OptimalControlProblems.jl - Curated (under construction) collection of benchmark problems
Tags: optimization control benchmark nlp gpu