Implementing inexact Newton solves in NonlinearSolve.jl

General Usage
1

I’d like to implement inexact Newton solves using NonlinearSolve.jl. The output from this code snippet (shown below) suggests that, by default, each Newton equation is solved to a fixed accuracy.

nlcache = init(
    prob,
    reltol=1e-3, abstol=1e-3,
    show_trace = Val(true),
    NewtonRaphson(linsolve = KrylovJL_MINRES(verbose=10, itmax=100)),
    )

for i in 1:10
    step!(nlcache)
end

Is it possible to modify nlcache within the loop to adaptively specify the accuracy of each linear solve?

Here’s the output:

┌ Warning: minres! doesn't support right preconditioning.
└ @ LinearSolve ~/.julia/packages/LinearSolve/B82H3/src/iterative_wrappers.jl:275
MINRES: system of size 202
    k      ‖r‖    ‖Aᴴr‖        β       cos       sin      ‖A‖     κ(A)    test1    test2  timer
    0  2.0e+00  0.0e+00  2.0e+00  -1.0e+00   0.0e+00  0.0e+00  0.0e+00  ✗ ✗ ✗ ✗  ✗ ✗ ✗ ✗  0.00s
   10  3.0e-02  3.3e-02  8.2e-01   6.0e-01   8.0e-01  1.7e+01  6.3e+00  1.9e-03  5.2e-02  0.00s
   20  1.9e-03  8.7e-03  4.4e+00   5.8e-01   8.2e-01  2.0e+01  8.5e+00  9.7e-05  1.8e-01  0.00s
   30  2.3e-04  2.0e-04  8.2e-01  -5.3e-01   8.5e-01  2.4e+01  9.2e+00  9.7e-06  3.2e-02  0.00s
   40  2.0e-05  2.2e-05  8.0e-01  -6.6e-01   7.5e-01  2.7e+01  9.2e+00  7.4e-07  3.0e-02  0.00s
   50  1.9e-06  1.8e-06  5.8e-01   6.9e-01   7.2e-01  3.0e+01  9.2e+00  6.7e-08  2.3e-02  0.00s
   60  1.5e-07  1.5e-07  8.6e-01   5.6e-01   8.3e-01  3.2e+01  9.2e+00  4.7e-09  2.6e-02  0.00s

1        2.97303732e-02       9.81027009e-01      
┌ Warning: minres! doesn't support right preconditioning.
└ @ LinearSolve ~/.julia/packages/LinearSolve/B82H3/src/iterative_wrappers.jl:275
MINRES: system of size 202
    k      ‖r‖    ‖Aᴴr‖        β       cos       sin      ‖A‖     κ(A)    test1    test2  timer
    0  4.0e-01  0.0e+00  4.0e-01  -1.0e+00   0.0e+00  0.0e+00  0.0e+00  ✗ ✗ ✗ ✗  ✗ ✗ ✗ ✗  0.00s
   10  5.7e-04  8.9e-04  9.3e-01   7.0e-01   7.1e-01  3.0e+01  2.4e+01  1.0e-03  3.7e-02  0.00s
   20  4.6e-05  4.4e-05  6.2e-01   6.8e-01   7.4e-01  4.0e+01  3.5e+01  5.6e-05  1.7e-02  0.00s
   30  7.6e-06  7.6e-06  9.9e-01  -4.9e-01   8.7e-01  5.0e+01  3.5e+01  7.3e-06  1.8e-02  0.00s
   40  6.9e-07  1.2e-06  2.0e+00  -4.7e-01   8.8e-01  5.7e+01  3.7e+01  5.8e-07  2.8e-02  0.00s
   50  7.5e-08  9.5e-08  9.6e-01   6.7e-01   7.4e-01  6.4e+01  3.7e+01  5.7e-08  1.5e-02  0.00s

2        3.51646316e-03       2.07810848e-02      
┌ Warning: minres! doesn't support right preconditioning.
└ @ LinearSolve ~/.julia/packages/LinearSolve/B82H3/src/iterative_wrappers.jl:275
MINRES: system of size 202
    k      ‖r‖    ‖Aᴴr‖        β       cos       sin      ‖A‖     κ(A)    test1    test2  timer
    0  4.7e-02  0.0e+00  4.7e-02  -1.0e+00   0.0e+00  0.0e+00  0.0e+00  ✗ ✗ ✗ ✗  ✗ ✗ ✗ ✗  0.00s
   10  5.8e-05  5.6e-05  8.2e-01   5.9e-01   8.0e-01  2.5e+01  2.0e+01  8.5e-04  3.1e-02  0.00s
   20  4.4e-06  4.3e-06  6.7e-01   6.6e-01   7.5e-01  3.3e+01  2.6e+01  4.7e-05  2.2e-02  0.00s
   30  6.7e-07  6.4e-07  8.1e-01  -5.7e-01   8.2e-01  4.0e+01  2.6e+01  5.8e-06  2.0e-02  0.00s
   40  6.7e-08  1.2e-07  2.2e+00  -4.4e-01   9.0e-01  4.2e+01  2.9e+01  5.6e-07  3.8e-02  0.00s

3        6.82363870e-05       2.88556208e-03
2

what do you mean by adaptively set the accuracy?

3

In situations where the computation cost is primarily dominated by the Jacobian-vector products, further efficiency can be had by solving the Newton system inexactly. A common approach is to ask an iterative method (eg, MINRES, in my case) to obtain a search direction \Delta x\in\mathbb{R}^n that reduces the residual to first order:

\|J(x_k) \Delta x + r(x_k)\| \le \eta_k\|r(x_k)\|,

where r is the residual function, J is its Jacobian, and \eta_k\in(0,1) is a dynamic accuracy parameter.

In each step of the loop, I’d like to pass \eta_k to MINRES as an optimality tolerance. This iimplements the above criterion.