# Troubles using sparse autodifferentiation

**URL:** <https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337>\
**Category:** General Usage\
**Tags:** autodiff\
**Created:** [April 23, 2025, 4:17pm UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337 "2025-04-23T16:17:13Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![slwu89](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/slwu89/32/217323_2.png) [@slwu89](https://discourse.julialang.org/u/slwu89)\
**Post date:** [April 23, 2025, 4:17pm UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/1 "2025-04-23T16:17:13Z")

</div>

Hi all, I am following [Advanced tutorial · DifferentiationInterface.jl](https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterface/stable/tutorials/advanced/#Sparsity) and to try out some of the modern AD features it provides. My test case is here [FunWithSplines/workspace/MLE/core-5\_4\_1.jl at main · slwu89/FunWithSplines · GitHub](https://github.com/slwu89/FunWithSplines/blob/main/workspace/MLE/core-5_4_1.jl) where I need to compute a Hessian matrix which is very sparse. I am able to compute the dense Hessian no problem, but using the sparse backend results in the following error pasted below (the last few lines of the example compute the Hessian). I believe it is due to a comparison between numeric types in the underlying mathematical model. Can someone give any advice about how to still use sparse AD in this case?

For the curious, the model is from 5.4.1 of [https://www.cambridge.org/us/universitypress/subjects/statistics-probability/statistical-theory-and-methods/core-statistics](https://www.cambridge.org/us/universitypress/subjects/statistics-probability/statistical-theory-and-methods/core-statistics).

```julia
ERROR: TypeError: non-boolean (SparseConnectivityTracer.HessianTracer{SparseConnectivityTracer.DictHessianPattern{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}) used in boolean context
Stacktrace:
  [1] model_urchin_vol(ω::Float64, g::SparseConnectivityTracer.HessianTracer{…}, p::SparseConnectivityTracer.HessianTracer{…}, a::Float64)
    @ Main ~/Desktop/git/FunWithSplines/workspace/MLE/core-5_4_1.jl:41
  [2] nlyfb_urchin(b::Vector{SparseConnectivityTracer.HessianTracer{…}}, θ::Vector{Float64}, urchin::DataFrame)
    @ Main ~/Desktop/git/FunWithSplines/workspace/MLE/core-5_4_1.jl:59
  [3] (::DifferentiationInterface.FixTail{…})(args::Vector{…})
    @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/Yk2Kt/src/utils/context.jl:169
  [4] trace_function(::Type{…}, f::DifferentiationInterface.FixTail{…}, x::Vector{…})
    @ SparseConnectivityTracer ~/.julia/packages/SparseConnectivityTracer/litqn/src/trace_functions.jl:48
  [5] _hessian_sparsity(f::DifferentiationInterface.FixTail{…}, x::Vector{…}, ::Type{…})
    @ SparseConnectivityTracer ~/.julia/packages/SparseConnectivityTracer/litqn/src/trace_functions.jl:115
  [6] hessian_sparsity
    @ ~/.julia/packages/SparseConnectivityTracer/litqn/src/adtypes_interface.jl:64 [inlined]
  [7] hessian_sparsity_with_contexts
    @ ~/.julia/packages/DifferentiationInterface/Yk2Kt/ext/DifferentiationInterfaceSparseConnectivityTracerExt/DifferentiationInterfaceSparseConnectivityTracerExt.jl:62 [inlined]
  [8] prepare_hessian_nokwarg(::Val{…}, ::typeof(nlyfb_urchin), ::AutoSparse{…}, ::Vector{…}, ::Constant{…}, ::Constant{…})
    @ DifferentiationInterfaceSparseMatrixColoringsExt ~/.julia/packages/DifferentiationInterface/Yk2Kt/ext/DifferentiationInterfaceSparseMatrixColoringsExt/hessian.jl:29
  [9] hessian(::typeof(nlyfb_urchin), ::AutoSparse{…}, ::Vector{…}, ::Constant{…}, ::Constant{…})
    @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/Yk2Kt/src/fallbacks/no_prep.jl:103
 [10] top-level scope
    @ ~/Desktop/git/FunWithSplines/workspace/MLE/core-5_4_1.jl:104
Some type information was truncated. Use `show(err)` to see complete types.

```

---

<div class="post-metadata">

**Author:** ![jClugstor](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jclugstor/32/32689_2.png) [@jClugstor](https://discourse.julialang.org/u/jClugstor)\
**Post date:** [April 23, 2025, 5:10pm UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/2 "2025-04-23T17:10:26Z")

</div>

The `TracerSparsityDetector` tracers don’t work with comparison operators, you might need to use `TracerLocalSparsityDetector`, [GitHub - adrhill/SparseConnectivityTracer.jl: Fast operator-overloading Jacobian & Hessian sparsity detection.](https://github.com/adrhill/SparseConnectivityTracer.jl?tab=readme-ov-file#local-tracing) .

---

<div class="post-metadata">

**Author:** ![slwu89](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/slwu89/32/217323_2.png) [@slwu89](https://discourse.julialang.org/u/slwu89)\
**Post date:** [April 23, 2025, 6:36pm UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/3 "2025-04-23T18:36:30Z")

</div>

Thanks! That works and I get a beautiful sparse hessian.

```julia
284×284 SparseArrays.SparseMatrixCSC{Float64, Int64} with 568 stored entries:
⎡⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠳⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎤
⎢⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠳⣄⠀⎥
⎢⢤⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠓⎥
⎢⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⠀⠀⎥
⎣⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⢦⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠑⢄⎦

```

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [April 26, 2025, 7:26am UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/4 "2025-04-26T07:26:01Z")

</div>

Note that to leverage the benefits of sparse AD, you should use preparation ahead of time and then reuse the preparation results. I saw from your stack trace that you’re not doing that, perhaps you’re just at an earlier step of the tutorial.  
However, the use of local sparsity tracers means that the resulting pattern theoretically depends on the input you provided. If you know that the pattern detected on the preparation input is actually input-independent, you can reuse it all you want, but the responsibility for that proof is yours. In any case, it is best to prepare with a random input than, say, an input full of zeros.

---

<div class="post-metadata">

**Author:** ![slwu89](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/slwu89/32/217323_2.png) [@slwu89](https://discourse.julialang.org/u/slwu89)\
**Post date:** [April 26, 2025, 12:36pm UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/5 "2025-04-26T12:36:22Z")

</div>

Hey thanks for following up! Yes in my full use case, I am using the `prepare_hessian` method to set up those objects.

Thanks for the advice on the local sparsity detector! Indeed after you pointed it out I do see that note in the docs [Getting Started · SparseConnectivityTracer.jl](https://adrianhill.de/SparseConnectivityTracer.jl/dev/#Local-tracing). That’s a good point, for “reasonable” parameter values based on the underlying biological model, it would have the same pattern but I can’t guarantee that across all possible values of the parameters. Based on some quick tests, the values around 0 should correspond to a less sparse pattern and values farther from 0 should be more sparse, as a strict subset (i.e. formerly dense entries are being set to 0). I’d need to do some work to definitively check.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [April 26, 2025, 12:59pm UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/6 "2025-04-26T12:59:48Z")

</div>

There are other alternatives if you can’t guarantee that local tracing is enough:

- you can use another sparsity detector, like `Symbolics.SymbolicsSparsityDetector()` (which will likely be slower but support more language constructs)
- you can (maybe) rewrite your function to replace comparison operators with `ifelse` statements

---

<div class="post-metadata">

**Author:** ![slwu89](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/slwu89/32/217323_2.png) [@slwu89](https://discourse.julialang.org/u/slwu89)\
**Post date:** [April 27, 2025, 4:25pm UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/7 "2025-04-27T16:25:17Z")

</div>

Thanks, for the example, I was able to use `Symbolics.SymbolicsSparsityDetector()` but only after replacing the comparison `<` clause between numeric variables with `Base.ifelse`. Thanks for the help; is the list of sparsity detectors here exhaustive? [Advanced features · DifferentiationInterface.jl](https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterface/stable/explanation/advanced/#AutoSparse-object)

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [April 27, 2025, 8:30pm UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/8 "2025-04-27T20:30:22Z")

</div>

> [@slwu89](#):
>
> Thanks, for the example, I was able to use `Symbolics.SymbolicsSparsityDetector()` but only after replacing the comparison `<` clause between numeric variables with `Base.ifelse`.

Thanks to this modification you might also be able to use `SCT.TracerSparsityDetector()` directly?

> [@slwu89](#):
>
> is the list of sparsity detectors here exhaustive?

To the best of my knowledge yes

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [April 27, 2025, 10:39pm UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/9 "2025-04-27T22:39:23Z")

</div>

You can also use Enzyme from within a Reactant.jl @compile (not available within DI) to leverage automatic sparsity propagation.

---

<div class="post-metadata">

**Author:** ![slwu89](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/slwu89/32/217323_2.png) [@slwu89](https://discourse.julialang.org/u/slwu89)\
**Post date:** [April 28, 2025, 3:30am UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/10 "2025-04-28T03:30:51Z")

</div>

Thanks. I’ll definitely check this out if I get into more serious applications of AD. For now I’m just poking around for fun at seeing what the ecosystem looks like for implementing statistics model of mid-level complexity (more than a call to fit a linear model, less than shape constrained splines embedded in a mixed model embedded in…)

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [April 28, 2025, 6:54am UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/11 "2025-04-28T06:54:59Z")

</div>

> [@wsmoses](#):
>
> You can also use Enzyme from within a [Reactant.jl](https://juliaregistries.github.io/General/packages/redirect_to_repo/Reactant) @compile (not available within DI) to leverage automatic sparsity propagation.

Is this what used to be called Spadina? Is it documented anywhere? Do you have an example code?

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [April 28, 2025, 7:12am UTC](https://discourse.julialang.org/t/troubles-using-sparse-autodifferentiation/128337/12 "2025-04-28T07:12:32Z")

</div>

No, Spadina is separate work enabled in Enzyme (though has not been given syntactic sugar in Enzyme.jl).

Reactant.jl (and Enzyme-JaX) contains a variety of sparsity propagation optimizations that apply to general codes, but are particularly useful for optimizing programs containing derivatives. Reactant docs are available online here: [Reactant.jl](https://enzymead.github.io/Reactant.jl/dev/), essentially just use @compile on the outermost function in your application (though see docs for more info)