I am puzzled by Fminbox decreasing
mu (the barier penalty term) to extremely low values. I do not quite understand why this happens and how I can control it.
I run something like (note in case it’s relevant that
my_obj_function involves simulation so that may introduce noise and hence keep the gradient “bouncing” around)
result = optimize( my_obj_function, theta_lower, theta_upper, theta_initial, Fminbox(BFGS(linesearch=BackTracking(order=3)), mufactor=1e-6), Optim.Options(outer_iterations=100, iterations=2000, show_trace=true, show_every=1, f_tol=1e-7, g_tol=1e-7, time_limit=10000))
this runs and runs for a long time, as shown below. Note: the current solution is extremely far from the barrier, and the inner loop (presumably) stops because the gradient falls below 1e-7. So, why continue to decrease mu?
How does Fminbox decide when to stop? And how many I control this?
(numbers below include barrier contribution) Iter Function value Gradient norm 14 1.663502e-05 1.795406e-07 * time: 11.428999900817871 # N.B: this is the time within the current loop not overall Exiting inner optimizer with x = [12.338068789784117, 66.91942126907855, 72.79429518182737, 67.47383322425802, -11.229420401921177, 52.853988010777655, -15.984116277134515, 10.016841794544156, -7.131259718778112, -15.626120659744577] Current distance to box: 12.3381 Decreasing barrier term μ. Fminbox iteration 33 -------------------- Calling inner optimizer with mu = 1.75391e-202