Julia frameworks for large-scale nonlinear least-squares optimization for computer vision

Hi, Julia newcomer here :slight_smile:

I’m working on learning both Julia and computer vision by implementing common vision algorithms like camera calibration in the Julia language. I’m at a point right now where I’m looking for a nonlinear least-squares framework designed to handle large-scale, sparse problems. Personally, I feel a little overwhelmed by all the options Julia offers (Optim.jl, JSO, NLSolve, JuMP, Minpack, LsqFit, etc…) and I’m wondering if anyone else has done camera calibration/structure-from-motion/bundle adjustment problems in Julia and is willing to share their experiences? It seems to me that the listed libraries above seem to share a lot of common functionality, and its not totally clear to me how one goes about choosing a particular framework?

So far https://github.com/matthieugomez/LeastSquaresOptim.jl seems the most promising option that best fits my use case, but I was wondering if anyone else has used JuMP, Optim, etc… before for my kind of NLS problem?

Thanks,
Sam

2 Likes

@yakir12 not so much for the optimization bit but the camera calibration bit!

2 Likes

I’ve been needing a camera calibration package in Julia since Julia v0.3. I use Matlab’s toolbox from within Julia, which worked fine, but is obviously not ideal. In any case, just yesterday my CameraCalibration.jl package got registered. I’ll make a PA here next week, but you can go check it out here if you like: https://github.com/yakir12/CameraCalibrations

I believe that there’s a great opportunity for an awesome camera calibration package in Julia, you can read some of the related activity here: Camera calibration.

Exciting!

2 Likes

I wrote two packages that together can do bundle adjustment (and other such optimizations) very efficiently:

NLLSsolver.jl - A non-linear least squares solver that supports sparse problems, auto-differentiation, and robustified costs.

VisualGeometryOptimization.jl - This package constructs visual geometry optimization problems that can then be solved by NLLSsolver.

I am seeking collaborators to further extend the functionality of the second package.

1 Like