# Is there a way to get more type information in stack traces / what heuristics are

**URL:** <https://discourse.julialang.org/t/is-there-a-way-to-get-more-type-information-in-stack-traces-what-heuristics-are/55226>\
**Category:** General Usage\
**Created:** [February 13, 2021, 8:17pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-get-more-type-information-in-stack-traces-what-heuristics-are/55226 "2021-02-13T20:17:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![BridgeBot](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bridgebot/32/21491_2.png) [@BridgeBot](https://discourse.julialang.org/u/BridgeBot)\
**Post date:** [February 13, 2021, 8:17pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-get-more-type-information-in-stack-traces-what-heuristics-are/55226/1 "2021-02-13T20:17:50Z")

</div>

Is there a way to get more type information in stack traces / what heuristics are used to decide to strip them?  
Debugging chains of functions whose behavior is determined by types of the arguments is much harder when I don’t know what the types are. Constructing objects of the appropriate types to debug would be easy, if I just knew what types to construct.

Note that the original poster on Slack cannot see your response here on Discourse. Consider _transcribing the appropriate answer back to Slack_, or pinging the poster here on Discourse so they can _follow this thread_.  
[(Original message :slack:)](https://julialang.slack.com/archives/C6A044SQH/p1613247447240100?thread_ts=1613247447.240100&cid=C6A044SQH) [(More Info)](https://github.com/JuliaCommunity/SlackBridge)

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [February 13, 2021, 11:26pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-get-more-type-information-in-stack-traces-what-heuristics-are/55226/2 "2021-02-13T23:26:35Z")

</div>

In case anyone finds this while searching for an answer:

1. I don’t know of a good way to get more details in the stack trace, but
2. If you’ve checked out the package for development, you can edit the functions to `@show` any information you want about the arguments and rerun the code thanks to Revise.
3. Use Cthulhu to descend into the function.

I’d still like to immediately have more details, but newcomers may find suggestions 2 and 3 helpful.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [February 15, 2021, 12:15pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-get-more-type-information-in-stack-traces-what-heuristics-are/55226/3 "2021-02-15T12:15:34Z")

</div>

If you’re using an up-to-date version of Cthulhu, you can now use `ascend` to inspect stack traces. It takes a little time to learn how to use Cthulhu but it is wickedly powerful, and I recommend investing the time to learn it for any Julia programmer who is (1) comfortable with the language, (2) understands the basic ideas of types and at least the general concept of type-inference, and (3) wants to know how to write Julia code of high quality. (EDIT: @Elrod , didn’t realize you were the asker of the question, I know you know Cthulhu!)

There’s a demo using Cthulhu for stacktraces on Cthulhu’s REAMDE:

> **[GitHub - JuliaDebug/Cthulhu.jl: The slow descent into madness](https://github.com/JuliaDebug/Cthulhu.jl)**
>
> The slow descent into madness. Contribute to JuliaDebug/Cthulhu.jl development by creating an account on GitHub.

which I’ve mostly reproduced here to provide more insight into the stack traces themselves.

Let’s take this case:

```julia
julia> sqrt(-1)
ERROR: DomainError with -1.0:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
Stacktrace:
 [1] throw_complex_domainerror(f::Symbol, x::Float64)
   @ Base.Math ./math.jl:33
 [2] sqrt
   @ ./math.jl:582 [inlined]
 [3] sqrt(x::Int64)
   @ Base.Math ./math.jl:608
 [4] top-level scope
   @ REPL[1]:1

```

You see type information in frames 1 and 3 but not in 2. That’s because of inlining, in the compiled code the code from math.jl:582 gets “pasted in” to its caller at math.jl:608.

If you catch the raw backtrace:

```julia
julia> bt = try
           sqrt(-1)
       catch
           catch_backtrace()
       end
37-element Vector{Union{Ptr{Nothing}, Base.InterpreterIP}}:
 Ptr{Nothing} @0x00007ff30fb399d9
 Ptr{Nothing} @0x00007ff2df803816
 Ptr{Nothing} @0x00007ff2df803846
 Ptr{Nothing} @0x00007ff31d5324fb
 Ptr{Nothing} @0x00007ff31d54c8ab
 Ptr{Nothing} @0x00007ff31d54c3f4
 Ptr{Nothing} @0x00007ff31d54d4f2
 Ptr{Nothing} @0x00007ff31d54d5c7
 Ptr{Nothing} @0x00007ff31d54dd16
 Base.InterpreterIP in top-level CodeInfo for Main at statement 1
 Ptr{Nothing} @0x00007ff31d56a586
 Ptr{Nothing} @0x00007ff31d56b23f
 Ptr{Nothing} @0x00007ff31d56b23f
 Ptr{Nothing} @0x00007ff31d56c3a1
 Ptr{Nothing} @0x00007ff30f8db031
 Ptr{Nothing} @0x00007ff31d5324fb
 Ptr{Nothing} @0x00007ff30f610e7c
 Ptr{Nothing} @0x00007ff30f611697
 Ptr{Nothing} @0x00007ff30f61178d
 Ptr{Nothing} @0x00007ff30f61fcd2
 Ptr{Nothing} @0x00007ff30f61fda8
 Ptr{Nothing} @0x00007ff31d5324fb
 Ptr{Nothing} @0x00007ff30f76e67a
 Ptr{Nothing} @0x00007ff30f76e718
 Ptr{Nothing} @0x00007ff31d5324fb
 Ptr{Nothing} @0x00007ff31d543690
 Ptr{Nothing} @0x00007ff31d543f65
 Ptr{Nothing} @0x00007ff30f841668
 Ptr{Nothing} @0x00007ff30f854fa1
 Ptr{Nothing} @0x00007ff30f855fe0
 Ptr{Nothing} @0x00007ff30f856155
 Ptr{Nothing} @0x00007ff31d5324fb
 Ptr{Nothing} @0x00007ff31d58da99
 Ptr{Nothing} @0x00007ff31d58e4f3
 Ptr{Nothing} @0x000055662d6f908c
 Ptr{Nothing} @0x00007ff31dcef0b2
 Ptr{Nothing} @0x000055662d6f90cd

```

Then you can do this:

```julia
julia> using Cthulhu

julia> ascend(bt)
Choose a call for analysis (q to quit):
 > throw_complex_domainerror(::Symbol, ::Float64) at ./math.jl:33
       sqrt at ./math.jl:582 => sqrt(::Int64) at ./math.jl:608
         eval(::Module, ::Any) at ./boot.jl:360
           eval_user_input(::Any, ::REPL.REPLBackend) at /home/tim/src/julia-master/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:139
             repl_backend_loop(::REPL.REPLBackend) at /home/tim/src/julia-master/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:200
               start_repl_backend(::REPL.REPLBackend, ::Any) at /home/tim/src/julia-master/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:185
                 #run_repl#42(::Bool, ::typeof(REPL.run_repl), ::REPL.AbstractREPL, ::Any) at /home/tim/src/julia-master/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:317
                   run_repl(::REPL.AbstractREPL, ::Any) at /home/tim/src/julia-master/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:305
                     (::Base.var"#878#880"{Bool, Bool, Bool})(::Module) at ./client.jl:387
v #invokelatest#2 at ./essentials.jl:707 => invokelatest at ./essentials.jl:706 => run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:372

```

The inlining is reflected in the second line, the `=>`. This is an interactive menu; you can select a place to browse and descend in. If I select the line with lines 582/608, then I get

```julia
Choose caller of MethodInstance for throw_complex_domainerror(::Symbol, ::Float64) or proceed to typed code:
 • "math.jl", sqrt: lines [582]
   Browse typed code

```

Choosing the first opens the file in my editor. Choosing the second shows

```julia
Choose caller of MethodInstance for throw_complex_domainerror(::Symbol, ::Float64) or proceed to typed code:
   "math.jl", sqrt: lines [582]
 • Browse typed code

│ ─ %-1 = invoke sqrt(::Int64)::Float64
Variables
  #self#::Core.Const(sqrt)
  x::Int64

Body::Float64
    @ math.jl:608 within `sqrt'
1 ─ %1 = Base.Math.float(x)::Float64
│ %2 = Base.Math.sqrt(%1)::Float64
└── return %2

Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [v]erbose printing for warntype code, [d]ebuginfo, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %1 = float(::Int64)::Float64
   %2 = sqrt(::Float64)::Float64
   ↩

```

By default `ascend` turns off optimization, so the inlining hasn’t taken place yet; you can descend into the call `%2` and see what’s going on at line 582. Alternatively you can hit `o` to turn on optimization and see what the “real” code looks like, which includes inlining:

```julia
│ ─ %-1 = invoke sqrt(::Int64)::Float64
Variables
  #self#::Core.Const(sqrt)
  x::Int64

Body::Float64
    @ math.jl:608 within `sqrt'
   ┌ @ float.jl:206 within `float'
   │┌ @ float.jl:191 within `AbstractFloat'
   ││┌ @ float.jl:94 within `Float64'
1 ─│││ %1 = Base.sitofp(Float64, x)::Float64
│ └└└
│ @ math.jl:608 within `sqrt' @ math.jl:582
│ ┌ @ float.jl:371 within `<'
│ │ %2 = Base.lt_float(%1, 0.0)::Bool
│ └
└── goto #3 if not %2
2 ─ invoke Base.Math.throw_complex_domainerror(:sqrt::Symbol, %1::Float64)
└── unreachable
    @ math.jl:608 within `sqrt' @ math.jl:583
3 ─ %6 = Base.Math.sqrt_llvm(%1)::Float64
└── goto #4
    @ math.jl:608 within `sqrt'
4 ─ return %6

Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [v]erbose printing for warntype code, [d]ebuginfo, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %4 = invoke throw_complex_domainerror(::Symbol,::Float64)::Union{}
   ↩

```

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [February 16, 2021, 7:41am UTC](https://discourse.julialang.org/t/is-there-a-way-to-get-more-type-information-in-stack-traces-what-heuristics-are/55226/4 "2021-02-16T07:41:34Z")

</div>

> [@tim.holy](#):
>
> If you’re using an up-to-date version of Cthulhu, you can now use `ascend` to inspect stack traces.

Thanks for that detailed response! I confess I’ve spent so much time descending into madness, it hadn’t occurred to me to ascend out 😉 .  
(or check the docs new features)

Just ran into another case where the default printing is basically the opposite of what I want:

```julia
julia> @time(AmulB!(C1,A,B)); C1 ≈ A * B
ERROR: MethodError: no method matching tdot(::Ptr{Float64}, ::VectorizationBase.Unroll{2, 1, 9, 1, 8, 0x0000000000000000, 1, VectorizationBase.Unroll{1, 8, 3, 1, 8, 0x0000000000000000, 1, VectorizationBase.Unroll{1, 8, 3, 1, 8, 0x0000000000000000, 1, Tuple{ArrayInterface.StaticInt{0}, ArrayInterface.StaticInt{0}}}}}, ::Tuple{ArrayInterface.StaticInt{8}, Int64}, ::Tuple{Nothing, NTuple{4, Int64}})
Closest candidates are:
  tdot(::Ptr{T}, ::Tuple{A}, ::Tuple{B, Vararg}, ::Tuple{C, Vararg}) where {T, A, B, C} at /home/chriselrod/.julia/dev/VectorizationBase/src/strided_pointers/cartesian_indexing.jl:9
  tdot(::Ptr{T}, ::Tuple{A1, A2, Vararg}, ::Tuple{B1, B2, Vararg}, ::Tuple{C1, C2, Vararg}) where {T, A1, A2, B1, B2, C1, C2} at /home/chriselrod/.julia/dev/VectorizationBase/src/strided_pointers/cartesian_indexing.jl:18
  tdot(::Ptr{T}, ::Tuple{A}, ::Tuple{B, Vararg}) where {T, A, B} at /home/chriselrod/.julia/dev/VectorizationBase/src/strided_pointers/cartesian_indexing.jl:6
  ...
Stacktrace:
 [1] tdot
   @ ~/.julia/dev/VectorizationBase/src/strided_pointers/cse_stridemultiples.jl:91 [inlined]
 [2] linear_index
   @ ~/.julia/dev/VectorizationBase/src/strided_pointers/stridedpointers.jl:126 [inlined]
 [3] vstore!
   @ ~/.julia/dev/VectorizationBase/src/llvm_intrin/memory_addr.jl:1414 [inlined]
 [4] macro expansion
   @ ~/.julia/dev/LoopVectorization/src/reconstruct_loopset.jl:532 [inlined]
 [5] _avx_!(#unused#::Val{(0, 0, 0, 8, 64, 32, 64)}, #unused#::Val{(:numericconstant, Symbol("###zero###9###"), LoopVectorization.OperationStruct(0x0000000000000012, 0x0000000000000000, 0x0000000000000003, 0x0000000000000000, LoopVectorization.constant, 0x00, 0x01), :LoopVectorization, :getindex, LoopVectorization.OperationStruct(0x0000000000000023, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, LoopVectorization.memload, 0x01, 0x02), :LoopVectorization, :getindex, LoopVectorization.OperationStruct(0x0000000000000031, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, LoopVectorization.memload, 0x02, 0x03), :LoopVectorization, :vfmadd_fast, LoopVectorization.OperationStruct(0x0000000000000231, 0x0000000000000003, 0x0000000000000000, 0x0000000000020301, LoopVectorization.compute, 0x00, 0x01), :LoopVectorization, :identity, LoopVectorization.OperationStruct(0x0000000000000012, 0x0000000000000003, 0x0000000000000000, 0x0000000000000004, LoopVectorization.compute, 0x00, 0x01), :LoopVectorization, :setindex!, LoopVectorization.OperationStruct(0x0000000000000021, 0x0000000000000003, 0x0000000000000000, 0x0000000000000005, LoopVectorization.memstore, 0x03, 0x04))}, #unused#::Val{(LoopVectorization.ArrayRefStruct{:A, Symbol("##vptr##_A")}(0x0000000000000101, 0x0000000000000203, 0x0000000000000000), LoopVectorization.ArrayRefStruct{:B, Symbol("##vptr##_B")}(0x0000000000000101, 0x0000000000000301, 0x0000000000000000), LoopVectorization.ArrayRefStruct{:C, Symbol("##vptr##_C")}(0x0000000000000101, 0x0000000000000201, 0x0000000000000000))}, #unused#::Val{(0, (), (), (), (), ((1, LoopVectorization.IntOrFloat),), ())}, #unused#::Val{(:n, :m, :k)}, _vargs::Tuple{Tuple{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}, ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}, ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}, Tuple{VectorizationBase.StridedPointer{Float64, 2, 1, 0, (1, 2), Tuple{ArrayInterface.StaticInt{8}, Int64}, Tuple{ArrayInterface.StaticInt{1}, ArrayInterface.StaticInt{1}}}, VectorizationBase.StridedPointer{Float64, 2, 1, 0, (1, 2), Tuple{ArrayInterface.StaticInt{8}, Int64}, Tuple{ArrayInterface.StaticInt{1}, ArrayInterface.StaticInt{1}}}, VectorizationBase.StridedPointer{Float64, 2, 1, 0, (1, 2), Tuple{ArrayInterface.StaticInt{8}, Int64}, Tuple{ArrayInterface.StaticInt{1}, ArrayInterface.StaticInt{1}}}}})
   @ LoopVectorization ~/.julia/dev/LoopVectorization/src/reconstruct_loopset.jl:532
 [6] AmulB!(C::Matrix{Float64}, A::Matrix{Float64}, B::Matrix{Float64})
   @ Main ./REPL[2]:2
 [7] top-level scope
   @ ./timing.jl:206 [inlined]
 [8] top-level scope
   @ ./REPL[5]:0

```

I know the types of the arguments to the top level function, don’t really care for the massive type dump for the `_avx_!` call, but from there it’s suddenly a lot more interesting.  
I’ve been making a lot of changes to codegen, both on the LoopVectorization and VectorizationBase side.  
So, the question is, is the dispatch chain going as I intend it to?

Yet it’s from here that suddenly all type information is missing. Of course, this also corresponds to which functions are inlined (and @mcabbott also suggested starting Julia with inlining disbaled while debugging).

Taking this example:

```julia
julia> bt = try
          AmulB!(C1,A,B)
       catch
           catch_backtrace()
       end;

julia> ascend(bt)

```

I now select `tdot`, the first option, and get

```julia
^ %7959 = False()::Core.Const(ArrayInterface.False())
   %7961 = StaticInt()::Core.Const(static(64))
   %7962 = vstore!(…,…,…,…,…,…,…,…)::Union{}
   %7980 = StaticInt()::Core.Const(static(0))
   %7982 = StaticInt()::Core.Const(static(8))
   %7984 = gesp(::typename(VectorizationBase.OffsetPrecalc){…},::typename(Tuple){…})
   %7986 = StaticInt()::Core.Const(static(0))
   %7988 = StaticInt()::Core.Const(static(8))
   %7990 = gesp(::typename(VectorizationBase.OffsetPrecalc){…},::typename(Tuple){…})
 • ↩

```

This is a little unweildy, when inlining is disabled, `_avx_!` contains 7990 function calls! With optimizations on, most of them are no-ops, and the rest will inline into either builtins or `llvmcall`s.

This is less than ideal, I’d like to be able to start ascending from `tdot` itself, ascending up to `linear_index` and `vstore!` and ultimately `_avx_!` if I desire. These are both much smaller, and where the problem occured, and would save me from _possibly_ having to wade through almost 8000 function calls. In this case, getting to `_avx_!` from top level is easy (but only with the help of Cthulhu, there’s no reasonable way to introspect into LoopVectorization without Cthulhu.jl!), but finding `vstore!` _potentially_ isn’t.

Emphasis on “potentially” and “possibly”, because as you probably noticed if looking at the snippet of functions:

```julia
   %7962 = vstore!(…,…,…,…,…,…,…,…)::Union{}

```

There is a `vstore!(...)::Union{}`, meaning I do immediately see the problem function failing to return correctly (`::Union{}`), and can descend to poke around.

So maybe it isn’t a big deal, but

> [@tim.holy](#):
>
> The inlining is reflected in the second line, the `=>` . This is an interactive menu; you can select a place to browse and descend in. If I select the line with lines 582/608, then I get
> 
> ```julia
> Choose caller of MethodInstance for throw_complex_domainerror(::Symbol, ::Float64) or proceed to typed code:
> • "math.jl", sqrt: lines [582]
> Browse typed code
> 
> ```

Maybe it’d be more convenient, when picking that option, to start at `"math.jl", sqrt: lines [582]` so you can ascend (`↩`) to reach line 608, instead of starting at 602 with the ability to descend to 582?

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [February 16, 2021, 10:01am UTC](https://discourse.julialang.org/t/is-there-a-way-to-get-more-type-information-in-stack-traces-what-heuristics-are/55226/5 "2021-02-16T10:01:11Z")

</div>

I sympathize with the challenges. In general the problem is this: when you don’t have the argument types, there’s nothing to `@descend` into. So you have to start somewhere you have all the types and then work your way downward.

During the inlining phase, the call is of course made with particular argument types, but that information gets discarded in the stack traces. I wonder if you might have more luck looking at `@snoopi_deep` and seeing what got inferred? As long as you can isolate your error to a relatively small chain of calls. You’ll have to put the operation inside a `try/catch` in order to retrieve the snooping info. You’ll be interested in the final child, which should be the one terminated by the error, unless that code was inferred on a previous call.