I wrote two packages that together can do bundle adjustment (and other such optimizations) very efficiently:
NLLSsolver.jl - A non-linear least squares solver that supports sparse problems, auto-differentiation, and robustified costs.
VisualGeometryOptimization.jl - This package constructs visual geometry optimization problems that can then be solved by NLLSsolver.
I am seeking collaborators to further extend the functionality of the second package.