# Why isn't SnoopPrecompile being able to fully compile the calls in this case?

**URL:** <https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849>\
**Category:** Performance\
**Tags:** package, snoopcompile\
**Created:** [January 12, 2023, 7:38am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849 "2023-01-12T07:38:01Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [January 12, 2023, 7:38am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/1 "2023-01-12T07:38:01Z")

</div>

I have [a PR](https://github.com/JuliaLinearAlgebra/BandedMatrices.jl/pull/283/files) to `BandedMatrices` with the following `SnoopPrecompile` instructions:

```julia
@precompile_setup begin
	vs = ([1.0], Float32[1.0], ComplexF32[1.0], ComplexF64[1.0])
	Bs = Any[BandedMatrix(0 => v) for v in vs]
	@precompile_all_calls begin
		for B in Bs, op in (+, -, *)
			op(B, B)
		end
		for (v, B) in zip(vs, Bs)
			B * v
		end
	end
end

```

The compilation fully [removes the TTFX](https://github.com/JuliaLinearAlgebra/BandedMatrices.jl/pull/283#issuecomment-1379893550) overhead in the `+`, `-` and `*` operations, but doesn’t in the `B * v` one. I wonder why it doesn’t work for the matrix-vector product (while it does for the matrix-matrix one)? Both set of calls are ultimately handled by `ArrayLayouts`, and would presumably descend to BLAS calls.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 12, 2023, 10:47am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/2 "2023-01-12T10:47:21Z")

</div>

Have you checked for invalidations? See the [script here](https://timholy.github.io/SnoopCompile.jl/stable/tutorial/#Cut-to-the-Chase:-A-copy-paste-analysis-of-invalidations) (while the plotting is very nice, it’s optional, the key info is in `staletrees` and you can just `display` it).

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [January 12, 2023, 11:15am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/3 "2023-01-12T11:15:44Z")

</div>

`staletrees` seems empty in this case

```julia
julia> using SnoopCompileCore

julia> invalidations = @snoopr using BandedMatrices;

julia> tinf = @snoopi_deep begin
           B = BandedMatrix(0 => zeros(10));
           v = rand(size(B,2));
           B * v
       end;

julia> using SnoopCompile

julia> trees = invalidation_trees(invalidations);

julia> staletrees = precompile_blockers(trees, tinf)
SnoopCompile.StaleTree[]

```

The most offensive method leading to invalidation seems to be

```julia
julia> trees[end]
inserting all(f::Function, x::FillArrays.AbstractFill) @ FillArrays ~/.julia/packages/FillArrays/o1UXZ/src/FillArrays.jl:590 invalidated:
   backedges: 1: superseding all(f::Function, a::AbstractArray; dims) @ Base reducedim.jl:1007 with MethodInstance for all(::TOML.Internals.Printer.var"#1#2", ::AbstractVector) (256 children)

```

and I wonder if there’s a way to resolve this in any case?

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 12, 2023, 11:18am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/4 "2023-01-12T11:18:25Z")

</div>

If `staletrees` is empty then (barring bugs in SnoopCompile) presumably the invalidations are not the explanation to the failure to use cached code for `*`. The `all` invalidation looks like it only affects TOML parsing, and will likely be fixed by [use invokelatest to prevent invalidations in TOML by KristofferC · Pull Request #48083 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/48083).

What does `ProfileView.view(flamegraph(tinf))` show? You can left-click on the base of each flame to see the entry point for type-inference. You can also see the list with `tinf.children`.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [January 12, 2023, 11:20am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/5 "2023-01-12T11:20:20Z")

</div>

Ah no, my bad, I was on the master branch. On the PR branch including SnoopPrecompile, staletrees isn’t empty.

```julia
julia> using SnoopCompileCore

julia> invalidations = @snoopr using BandedMatrices;
[Info: Precompiling BandedMatrices [aae01518-5342-5314-be14-df237901396f]

julia> tinf = @snoopi_deep begin
           B = BandedMatrix(0 => zeros(10));
           v = rand(size(B,2));
           B * v
       end;

julia> using SnoopCompile

julia> trees = invalidation_trees(invalidations);

julia> staletrees = precompile_blockers(trees, tinf)
3-element Vector{SnoopCompile.StaleTree}:
 inserting eltype(::Type{T}) where T<:StableIndexedCursor @ AbstractTrees ~/.julia/packages/AbstractTrees/x9S7q/src/cursors.jl:332 invalidated:
   mt_backedges: 1: MethodInstance for eltype(::AbstractVector{Float64}) at depth 1 with 17 children blocked InferenceTimingNode: 0.000295/0.411388 on *(::BandedMatrix{Float64, Matrix{Float64}, Base.OneTo{Int64}}, ::Vector{Float64}) with 1 direct children

 inserting eltype(::Type{T}) where T<:StableCursor @ AbstractTrees ~/.julia/packages/AbstractTrees/x9S7q/src/cursors.jl:295 invalidated:
   mt_backedges: 1: MethodInstance for eltype(::AbstractVector{Float64}) at depth 1 with 17 children blocked InferenceTimingNode: 0.000295/0.411388 on *(::BandedMatrix{Float64, Matrix{Float64}, Base.OneTo{Int64}}, ::Vector{Float64}) with 1 direct children

 inserting eltype(::Type{<:AbstractTrees.TreeIterator{T}}) where T @ AbstractTrees ~/.julia/packages/AbstractTrees/x9S7q/src/iteration.jl:68 invalidated:
   mt_backedges: 1: MethodInstance for eltype(::AbstractVector{Float64}) at depth 1 with 17 children blocked InferenceTimingNode: 0.000295/0.411388 on *(::BandedMatrix{Float64, Matrix{Float64}, Base.OneTo{Int64}}, ::Vector{Float64}) with 1 direct children

```

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 12, 2023, 11:24am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/6 "2023-01-12T11:24:09Z")

</div>

It looks like the `*` method that gets used for a `BandedMatrix` has some inference failures. If those are fixable, it should resolve the invalidations. And of course you might also get a performance boost as well.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [January 12, 2023, 11:33am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/7 "2023-01-12T11:33:56Z")

</div>

Indeed, Cthulhu seems to hint at something

```julia
• %407 = invoke _banded_muladd!(::Float64,::SubArray{…},::Array{…},::Float64,::SubArray{…})::Any

```

but frustatingly I hit [`UndefVarError`: `mi` not defined · Issue #333 · JuliaDebug/Cthulhu.jl · GitHub](https://github.com/JuliaDebug/Cthulhu.jl/issues/333), so I guess it’ll take a while to get to the bottom of this. Thanks a lot for looking into this!

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 12, 2023, 11:41am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/8 "2023-01-12T11:41:09Z")

</div>

Can you just change line 476 to `additional_descend(get_mi(curs))`?

For a good PR the hard part will be capturing a test, but this seems likely to fix it.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [January 12, 2023, 11:58am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/9 "2023-01-12T11:58:51Z")

</div>

Thanks, that helps! The issue might be that `_banded_muladd` is recursive, which leads to inference failure

```julia
function _banded_muladd!(α::T, A, x::AbstractVector, β, y) where T
    m, n = size(A)
    (length(y) ≠ m || length(x) ≠ n) && throw(DimensionMismatch("*"))
    l, u = bandwidths(A)
    if -l > u # no bands
        _fill_lmul!(β, y)
    elseif l < 0
        _banded_muladd!(α, view(A, :, 1-l:n), view(x, 1-l:n), β, y)
    elseif u < 0
        y[1:-u] .= zero(T)
        _banded_muladd!(α, view(A, 1-u:m, :), x, β, view(y, 1-u:m))
        y
    else
        _banded_gbmv!('N', α, A, x, β, y)
    end
end

```

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 12, 2023, 12:19pm UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/10 "2023-01-12T12:19:25Z")

</div>

Good guess, as recursive calls that change some of the types present real challenges for inference. If in the outer call you can predict which branch the inner call will take (i.e., the output of `bandwidths` is predictable based on the `view`), then presumably this is cleanly fixable. If not, a good bandaid might be to make self-calls `Base.invokelatest(_banded_muladd!, α, ...)`.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [January 13, 2023, 7:36am UTC](https://discourse.julialang.org/t/why-isnt-snoopprecompile-being-able-to-fully-compile-the-calls-in-this-case/92849/11 "2023-01-13T07:36:49Z")

</div>

This does seem cleanly fixable, and I’ve [created a PR](https://github.com/JuliaLinearAlgebra/BandedMatrices.jl/pull/293). This fix resolves the precompilation issue completely.