# How to extract intermediate results from GMRES of Krylov.jl

**URL:** <https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019>\
**Category:** Optimization (Mathematical)\
**Created:** [September 9, 2022, 8:04pm UTC](https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019 "2022-09-09T20:04:44Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jinml](https://avatars.discourse-cdn.com/v4/letter/j/91b2a8/32.png) [@jinml](https://discourse.julialang.org/u/jinml)\
**Post date:** [September 9, 2022, 8:04pm UTC](https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019/1 "2022-09-09T20:04:44Z")

</div>

Hi, I am using `gmres` from `Krylov.jl` as follows

```julia
x,stat=gmres(A,b,itmax=3000,memory=500,history=true)

```

which will return some information about the iteration, e.g., `stat.residuals`. Is there a way to also return the intermediate `x`’s for diagnostic purposes? By changing the `verbose` keyword does not seem to do it. I looked through documentation [here](https://juliasmoothoptimizers.github.io/Krylov.jl/stable/solvers/unsymmetric/#Krylov.gmres) but couldn’t find anything relevant.  
Does anyone have experience with this GMRES implementation?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 9, 2022, 8:11pm UTC](https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019/2 "2022-09-09T20:11:36Z")

</div>

you could register a callback. the reason this doesn’t exist by default is that saving the intermediate values is pretty expensive.

---

<div class="post-metadata">

**Author:** ![jinml](https://avatars.discourse-cdn.com/v4/letter/j/91b2a8/32.png) [@jinml](https://discourse.julialang.org/u/jinml)\
**Post date:** [September 9, 2022, 8:47pm UTC](https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019/3 "2022-09-09T20:47:37Z")

</div>

Thank you for your answer. I looked at [Callbacks](https://juliasmoothoptimizers.github.io/Krylov.jl/stable/callbacks/) but cannot see how the callback function can access intermediate variables from the solver. Can you elaborate a little more?

---

<div class="post-metadata">

**Author:** ![amontoison](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amontoison/32/218741_2.png) [@amontoison](https://discourse.julialang.org/u/amontoison)\
**Post date:** [September 9, 2022, 10:26pm UTC](https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019/4 "2022-09-09T22:26:39Z")

</div>

Hi @jinml!

`FOM`, `GMRES` and `GPMR` are the only methods in Krylov.jl that compute the solution `x` at the last iteration.  
We can’t update `x` because `x = V * y` where `y` is the solution of a triangular system `Ry = z`. At each iteration, the coefficients of `y` are different if we perform the backward substitution.

However, you can create a callback that compute each `xk`. The initial goal of a callback is to add new stopping conditions but you can also use it to see the current state of the KrylovSolver (workspace).

```julia
T = eltype(b)
global X = Vector{T}[]

function jinml_callback(solver)
   z = solver.z
   k = solver.inner_iter
   nr = sum(1:k)
   V = solver.V
   R = solver.R
    y = copy(z)
    for i = k : -1 : 1
      pos = nr + i - k # position of rᵢ.ₖ
      for j = k : -1 : i+1
        y[i] = y[i] - R[pos] * y[j] # yᵢ ← yᵢ - rᵢⱼyⱼ
        pos = pos - j + 1 # position of rᵢ.ⱼ₋₁
      end
      y[i] = y[i] / R[pos] # yᵢ ← yᵢ / rᵢᵢ
    end
    xk = sum(V[i] * y[i] for i = 1:k)
    push!(X, xk)
    return false
end

```

```julia
x, stats = gmres(A, b, itmax=3000, memory=500, history=true, callback=jinml_callback)
stats.residuals
residuals2 = [norm(b - A * X[i]) for i = 1:stats.niter]

```

---

<div class="post-metadata">

**Author:** ![jinml](https://avatars.discourse-cdn.com/v4/letter/j/91b2a8/32.png) [@jinml](https://discourse.julialang.org/u/jinml)\
**Post date:** [September 11, 2022, 3:05pm UTC](https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019/5 "2022-09-11T15:05:11Z")

</div>

Thanks @Oscar_Smith @amontoison for the detailed description. I was able to generate a convergence animation of a solution for your entertainment 😃  
 ![scatt2Intconv](https://global.discourse-cdn.com/julialang/original/3X/c/8/c81d538d45179c0e8b4edd34f660bbe1aeb47372.gif)

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 11, 2022, 3:06pm UTC](https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019/6 "2022-09-11T15:06:21Z")

</div>

That’s awesome!

---

<div class="post-metadata">

**Author:** ![amontoison](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amontoison/32/218741_2.png) [@amontoison](https://discourse.julialang.org/u/amontoison)\
**Post date:** [September 12, 2022, 6:59pm UTC](https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019/7 "2022-09-12T18:59:56Z")

</div>

It’s great @jinml! What do you plot exactly?

---

<div class="post-metadata">

**Author:** ![jinml](https://avatars.discourse-cdn.com/v4/letter/j/91b2a8/32.png) [@jinml](https://discourse.julialang.org/u/jinml)\
**Post date:** [September 12, 2022, 7:10pm UTC](https://discourse.julialang.org/t/how-to-extract-intermediate-results-from-gmres-of-krylov-jl/87019/8 "2022-09-12T19:10:05Z")

</div>

@amontoison It is the intermediate solutions to a wave equation using GMRES converging to the final result. The wave equation describes a wave traveling through two penetrable obstacles, and was focused by the convex shape of the obstacle.