# Maximum\_a\_posteriori not honoring abstol and reltol? Turing.jl

**URL:** <https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402>\
**Category:** General Usage\
**Tags:** optimization, turing\
**Created:** [October 23, 2025, 9:28pm UTC](https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402 "2025-10-23T21:28:09Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 23, 2025, 9:28pm UTC](https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402/1 "2025-10-23T21:28:09Z")

</div>

So, my student has a problem where he’s just been letting maximum\_a\_priori run for as long as it needs… which is taking like 2000+ seconds… I suggested to try abstol or reltol to make sure it wasn’t spending all its time on refining the last representable decimal place… Sure enough, using

maxtime=200:  
203.660970 seconds (10.85 M allocations: 6.851 GiB, 0.69% gc time)  
ModeResult with maximized lp of -22226.09

maxtime=100 we get:

103.657475 seconds (7.72 M allocations: 5.732 GiB, 0.38% gc time)  
ModeResult with maximized lp of -22228.50

60 seconds:

63.676032 seconds (6.35 M allocations: 5.243 GiB, 0.57% gc time)  
ModeResult with maximized lp of -22237.03

Obviously in these problems running for more than 100 ish seconds has highly diminishing returns to improving the log\_probability\_density, but there are many of these problems on different data sets and rather than guessing a sufficient time, it’d be better to say, stop when the LP doesn’t improve by more than say 0.5 or something (ie. abstol=0.5)

but when I set abstol it ALWAYS runs to the end of the maxtime it never stops early. Even with abstol=50 or more. This is using LBFGS() algorithm. same with reltol.

Are abstol and reltol not honored? Or maybe just when using this algorithm? or what?

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 24, 2025, 2:40pm UTC](https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402/2 "2025-10-24T14:40:33Z")

</div>

Thought this might be a known issue so someone would just pipe up, but apparently not?

Anyway I’ll work on a MWE and post it later today, we can see if in fact it’s a problem or just something specific to our code.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 24, 2025, 8:22pm UTC](https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402/3 "2025-10-24T20:22:02Z")

</div>

```julia

using Turing,Distributions,OptimizationOptimJL,PDMats

@model function mwe(mat)
    a ~ MvNormal(fill(0.0,20),mat)
end

covmat = PDiagMat(rand(Uniform(.1,10),20))

initval = fill(1.0,20)

vec = maximum_a_posteriori(mwe(covmat),Optim.LBFGS(); initial_params=initval,maxiters=50)
vec2 = maximum_a_posteriori(mwe(covmat),Optim.LBFGS(); initial_params=initval,abstol=12.0,maxiters=50)

norm(vec.values) |>display
norm(vec2.values) |> display

lpmax = logpdf(MvNormal(fill(0.0,20),covmat),fill(0.0,20))

diff = lpmax - vec.lp
diff2 = lpmax - vec2.lp

```

Ok, so doing this, the second optimization does come to an earlier stop with a larger error in the lp value. So evidently abstol DID work. So, now I’ll show this to my student and see if there’s something we can do to make abstol work for us as well.

I’ll come back and report what we find.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 27, 2025, 9:54pm UTC](https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402/4 "2025-10-27T21:54:29Z")

</div>

Using my MWE on the current version of Turing works, but I think on the older version of Turing that was in our Manifest.toml from a while back it DOES NOT work. At least, my colleague reports that the MWE doesn’t stop early using abstol with that older version. I’m not sure what version that was but it might be that a bug got fixed in the interim between now and whenever in the past we added Turing to our project.

So, if you’re using optimization to find max a-priori estimates with Turing, use at least a late 2025 era version of all relevant packages.

---

<div class="post-metadata">

**Author:** ![khoffie](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/khoffie/32/215904_2.png) [@khoffie](https://discourse.julialang.org/u/khoffie)\
**Post date:** [October 28, 2025, 7:44am UTC](https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402/5 "2025-10-28T07:44:08Z")

</div>

dlakelan’s colleague here. Apologies for the confusion, but the problem seems not related to the Turing version. Rather abstol sometimes works and sometimes doesn’t, depending on the problem. Setting `abstol = 12.0` does _not_ stop early, if the covariance matrix is highly correlated.

```julia-auto
using Turing, LinearAlgebra, Optim

N = 20

@model function mwe(mat)
    a ~ MvNormal(fill(0.0, size(mat)[1]), mat)
end

covmat1 = Diagonal(rand(Uniform(.1, 10), N))

x = rand(Uniform(.1, 10), N);
covmat2 = x * x' + .01I

initval = fill(10.0, N);

##### model with covariance matrix 1
vec = maximum_a_posteriori(mwe(covmat1), Optim.LBFGS(); initial_params = initval);
vec2 = maximum_a_posteriori(mwe(covmat1), Optim.LBFGS(); initial_params = initval, abstol = 12.0);

norm(vec.values) |>display
norm(vec2.values) |> display

lpmax = logpdf(MvNormal(fill(0.0, N), covmat1), fill(0.0, N));
diff = lpmax - vec.lp
diff2 = lpmax - vec2.lp

#### model with covariance matrix 2
vec = maximum_a_posteriori(mwe(covmat2), Optim.LBFGS(); initial_params = initval);
vec2 = maximum_a_posteriori(mwe(covmat2), Optim.LBFGS(); initial_params = initval, abstol = 12.0);

lpmax = logpdf(MvNormal(fill(0.0, N),covmat2), fill(0.0, N));
diff = lpmax - vec.lp
diff2 = lpmax - vec2.lp

```

While the first does stop early, the second does not

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 29, 2025, 6:00pm UTC](https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402/6 "2025-10-29T18:00:37Z")

</div>

So, I ran the code adding `show_trace=true` and upping the `abstol = 12000.0` and in fact the correlated case does terminate early with the abstol by 1 iteration… basically this fairly simple quadratic problem converges really quickly so is not a great test case, but abstol DOES work, with current versions of Turing etc:

without the abstol, it does 0,1,2,3,4 iterations with the abstol it stops after iteration number 3

I don’t know what abstol is actually measuring. Here are the iteration, function value, and gradient norm calcs:

```julia-auto
     0 3.222598e+07 3.314722e+04
     1 9.291964e+02 8.245096e-01
     2 -2.231984e+01 1.105896e-03
     3 -2.231984e+01 3.099994e-08
     4 -2.231984e+01 7.543506e-24

```

So if abstol is 12000.0 I would think it should stop at iteration 2 where the function improvement was about 951.5 instead it does iteration 3 and stops there where the function improvement is 0 to 7 decimal places. while without the abstol it stops at iteration 4 where the function improvement was 0 to many many decimal places and the gradient was 0 to 24 decimal places.

---

<div class="post-metadata">

**Author:** ![penelopeysm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/penelopeysm/32/213172_2.png) [@penelopeysm](https://discourse.julialang.org/u/penelopeysm)\
**Post date:** [November 1, 2025, 12:14am UTC](https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402/7 "2025-11-01T00:14:25Z")

</div>

I don’t have the time right now to look into this (sorry), but I would suggest trying to cut out the Turing model, i.e. just use Optimization.jl directly on the function

```julia
mat = ...
dist = MvNormal(fill(0.0, size(mat)[1]), mat)
f(x) = -logpdf(dist, x)

```

and see if you still observe the same behaviour.

The `maximum_a_posteriori` function is pretty much a thin wrapper around this. Turing also makes sure the parameters are transformed to unconstrained space, but MvNormal is already unconstrained so that should be a no-op.

---

<div class="post-metadata">

**Author:** ![khoffie](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/khoffie/32/215904_2.png) [@khoffie](https://discourse.julialang.org/u/khoffie)\
**Post date:** [June 26, 2026, 6:45am UTC](https://discourse.julialang.org/t/maximum-a-posteriori-not-honoring-abstol-and-reltol-turing-jl/133402/8 "2026-06-26T06:45:38Z")

</div>

I checked again with new Turing version `0.45.0`. Setting `abstol = 12.0` does stop early now, for both covariance matrices. In old version it did not reliably stop the second optimization.

```julia-auto
using Turing, LinearAlgebra, Optim

@model function mwe(mat)
    a ~ MvNormal(fill(0.0, size(mat)[1]), mat)
end

N = 20
covmat1 = Diagonal(rand(Uniform(.1, 10), N))
x = rand(Uniform(.1, 10), N);
covmat2 = x * x' + .01I

function runmwe(model, cvm, abstol = 12.0)
    N = size(cvm, 1)
    inits = InitFromParams(VarNamedTuple(; a = fill(10.0, N)))
    r1 = maximum_a_posteriori(model(cvm), Optim.LBFGS(), initial_params = inits)
    r2 = maximum_a_posteriori(model(cvm), Optim.LBFGS(), initial_params = inits, abstol = abstol)

    lpmax = logpdf(MvNormal(fill(0.0, N), cvm), fill(0.0, N));
    res = (; norm1 = norm(values(r1.params)),
           norm2 = norm(values(r2.params)),
           lpdiff1 = lpmax - r1.lp,
           lpdiff2 = lpmax - r2.lp)
    return res
end

runmwe(mwe, covmat1)
runmwe(mwe, covmat2)

```

Not sure what caused the bug, was it in `Turing.jl` or `Optimization.jl`? But glad it works now.