# MAP optimization in Turing with LBFGS doesn't move at all

**URL:** https://discourse.julialang.org/t/map-optimization-in-turing-with-lbfgs-doesnt-move-at-all/118431
**Category:** Probabilistic Programming
**Tags:** optimization, turing, autodiff
**Created:** [August 20, 2024, 10:10pm UTC](https://discourse.julialang.org/t/map-optimization-in-turing-with-lbfgs-doesnt-move-at-all/118431 "2024-08-20T22:10:32Z")
**Posts on this page:** 1
**Showing post:** 9

<div class="post-metadata">

### Author: ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)
#### Post date: [August 21, 2024, 4:52pm UTC](https://discourse.julialang.org/t/map-optimization-in-turing-with-lbfgs-doesnt-move-at-all/118431/9 "2024-08-21T16:52:51Z")

</div>

> [@dpo](#):
>
> I’m suggesting finite differences to check that the gradient is accurate, not necessarily to use as a substitute for the actual gradient.

Yes, I understood that, but thanks for making sure though.

Yes I too think that BOBYQA making not only progress but amazing progress while LBFGS didn’t suggests a gradient issue. It also suggests that the function shouldn’t be “too hard” to optimize because BOBYQA uses quadratic approximations and goes BANG right to the optimum, so I feel like this should be approximately a big normal distribution with probably a dense covariance matrix, lots of correlated parameters etc.

This is based on the maximum\_a\_posteriori interface in Turing, so underlying that it’s using Optimization.jl which specifies algorithms in terms of passing structs.

I can pass the LBFGS() struct, in which case I believe that Optimization.jl uses [Fminbox](https://discourse.julialang.org/t/optim-jl-do-all-methods-allow-box-constraints-should-all-work-without-them/10209/2) with LBFGS inside that… so Fminbox is basically adding penalties and then running LBFGS and then adjusting the penalties in a loop I think.

It’s entirely possible it’s this Fminbox that’s the problem? I don’t know. But that wouldn’t explain why LD\_LBFGS in NLopt doesn’t work. Though perhaps there it was not supporting box constraints… There are many things that are confusing to me.

One thing I may try is to fit with BOBYQA the full model with less constraints… find an optimum, perturb it a bit, and then run LBFGS from this perturbed optimum with NO constraints and see if it fits. perhaps Fminbox is the real problem and LBFGS would work fine if it were started close enough to the proper optimum.

I’ll do some experimentation, but a 12 hour marathon session yesterday means I probably will take today off. I’ll be back in the next few days with any info I can figure out though.

---

_[View the full topic](https://discourse.julialang.org/t/map-optimization-in-turing-with-lbfgs-doesnt-move-at-all/118431)._