Make Julia’s Error Codes Even Better Than Elm’s

[CameronBieganek]

@Ronis_BR

Any error inside a dynamics function of DifferetialEquations.jl leads to a huge text which is not relevant 99% of time

Your code example doesn’t drive the point home, because most of the extensive stacktrace is hidden via the horizontal scroll bar. A screenshot does a better job:

This has nothing to do with error messages though. The noise comes from the stacktrace. And the reason it is so bad is because for many of the (immutable) struct definitions in DifferentialEquations.jl, every single field is parameterized. Most of those fields don’t get used, so they’re just set to nothing, which is why you see an extraordinary amount of Nothing types in the stacktrace. It seems a bit wasteful to me.

[anon56330260]

I guess we need some post processing mechanisms to remove these intermediate frames from stacktraces. For example, we remove all but last one frames generated by external packages, and keep the frames in Main module and its submodules. That is, we only concern about errors caused by our codes, instead of errors caused by external packages.

[dlakelan]

This is exactly what AbbreviatedStackTraces.jl does. Not sure how it handles this example

[tecosaur]

For method errors, there are some basic improvements that can be had: GitHub - tecosaur/HelpfulErrors.jl

Stefan can correct me, but I don’t think there’s any performance impact there.

[lawless-m]

I tried to access a Vector{Int64} at index [0] but ...

* uses “I” for the compiler, which does have a more relatable tone IMHO

Relatable to who? I am not a compiler nor is the machine a person, it is an “it”.

You should never anthropomorphize computers, they hate that!

[jar1]

English is harder to understand when things are written in the passive voice. Giving the compiler a persona is admittedly a bit odd, but Elm shows it can work. It provides more room to the error-writer for explanation about what I was doing when I encountered a problem.

[mrufsvold]

Maybe the Royal We?

We attempted to access the array at index [3], but Our subjects failed to acquire it as the length of this array was only 2 at the moment at which thou requested the pursuit of the value of its field.

[Eben60]

@jar1

English is harder to understand when things are written in the passive voice.

Then I’d suggest “You”

You requested computer to access the array at index [0]. 
Please think twice before doing silly things like this again.

That’s actually correct, the problem is usually sitting in front of the computer.

OK, now seriously - first and foremost the error messages must correctly describe what happened.

I tried to access a Vector{Int64} at index [0]

is plainly wrong IMO - “trying” assumes a free will, which neither compiler no computer possess.

[lrnv]

What about You told me to access a vector at index 0, looser ! Please RTFM.

I like my computer to yell at me when I am clearly the error.

[Mason]

@StefanKarpinski

Yes, that is a much better error message. Unfortunately it also tends to make functions much slower even when errors aren’t thrown because it creates a complex branch that captures all values that are used in the error message, which forces those values to be materialized and/or heap allocated. Using a static error message, on the other hand, has negligible performance impact, which is why so many error messages are like that. We have some newish technology to improve this, such as LazyString, but while that can help, it doesn’t entirely eliminate the problem. I’m planning on trying to make some better error messages like you suggest to see how bad the impact is and evaluate what kind of compiler magic we need to make it not affect performance unbearably, but it almost certainly will require some compiler work.

Does this really require compiler work? We already have @noinline which is sufficient for this as far as I’m aware:

my_sqrt(x::Real) = x >= 0.0 ? sqrt(x) : my_sqrt_error(x);

@noinline my_sqrt_error(x) = throw(DomainError("my_sqrt requires inputs greater than 0, you gave $x which is less than 0"));

julia> @benchmark my_sqrt(x) setup=(x= 100*rand())
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  2.159 ns … 20.240 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     2.160 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   2.168 ns ±  0.192 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

   █          ▁                                               
  ▂█▁▁▁▁▁▁▁▁▁▂█▁▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▂▃ ▂
  2.16 ns        Histogram: frequency by time        2.21 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> @benchmark sqrt(x) setup=(x= 100*rand())
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  2.159 ns … 11.370 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     2.160 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   2.167 ns ±  0.107 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

   █          ▆                                 ▁          ▃ ▁
  ██▁▁▁▁▁▁▁▁▁▆█▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▄█ █
  2.16 ns      Histogram: log(frequency) by time     2.21 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

In fact, now that I look, it appears that whoever wrote the sqrt(::Float64) method in Base already knew and took advantage of this:

julia> @code_typed sqrt(1.0)
CodeInfo(
1 ─ %1 = Base.lt_float(x, 0.0)::Bool
└──      goto #3 if not %1
2 ─      invoke Base.Math.throw_complex_domainerror(:sqrt::Symbol, x::Float64)::Union{}
└──      unreachable
3 ─ %5 = Base.Math.sqrt_llvm(x)::Float64
└──      return %5
) => Float64

[StefanKarpinski]

@Mason

Does this really require compiler work? We already have @noinline which is sufficient for this as far as I’m aware

I really don’t want to have to create a separate function for every single error.

[Raf]

I mostly have @noinline errors in packages, and its not that awful? Also more straight forward and immediately actionable than compiler magic.

[Mason]

@StefanKarpinski

I really don’t want to have to create a separate function for every single error.

Then a simple macro would do the trick:

macro noinline_block(args...)
    captures = esc.(args[1:end-1])
    body = esc(args[end])
    quote
        @noinline block($(captures...),) = $body
        block($(captures...),)
    end
end

function f(x::Real)
    if x>= 0.0
        x + 1
    else
        @noinline_block x throw(DomainError("f requires inputs greater than 0, you gave $x which is less than 0"))
    end
end

julia> code_llvm(f, (Float64,), debuginfo=:none)
define double @julia_f_936(double %0) #0 {
top:
  %1 = fcmp ult double %0, 0.000000e+00
  br i1 %1, label %L5, label %L3

L3:                                               ; preds = %top
  %2 = fadd double %0, 1.000000e+00
  ret double %2

L5:                                               ; preds = %top
  %3 = call nonnull {}* @"j_#54#block_938"(double %0) #0
  call void @llvm.trap()
  unreachable
}

We could even make an @throw macro which automates this stuff.

[Raf]

Great idea for @throw, and looks pretty slick:

function f(x::Real)
    if x>= 0.0
        x + 1
    else
        @throw x DomainError("f requires inputs greater than 0, you gave $x which is less than 0")
    end
end

Solves both the performance and boilerplate issues.

[Mason]

Yeah, here’s what that macro definition would be:

macro throw(args...)
    captures = esc.(args[1:end-1])
    err = esc(args[end])
    quote
        @noinline throw_err($(captures...),) = throw($err)
        throw_err($(captures...),)
    end
end

Maybe this should go in a PR to base along with a spree of error message improvements?

[Mason]

Nah, macro hygiene takes care of that for you:

julia> macro throw(args...)
           captures = esc.(args[1:end-1])
           err = esc(args[end])
           quote
               @noinline throw_err($(captures...),) = throw($err)
               throw_err($(captures...),)
           end
       end;

julia> @macroexpand @throw x y
quote
    #= REPL[8]:5 =#
    var"#71#throw_err"(x) = begin
            $(Expr(:meta, :noinline))
            #= REPL[8]:5 =#
            Main.throw(y)
        end
    #= REPL[8]:6 =#
    var"#71#throw_err"(x)
end

The gensym would be warranted if I esc’d the whole expression though. But then I’d want to interpolate $throw and $Base.@inline.

[Mason]

One problem with the macro as written is that it pollutes the stacktrace. It’d be good to have a way to hide that.

julia> f(-1)
ERROR: DomainError with f requires inputs greater than 0, you gave -1 which is less than 0:

Stacktrace:
 [1] (::var"#104#throw_err#12")(x::Int64) # ⌉ 
   @ Main ./REPL[44]:5                    # |<------ Yuck.
 [2] macro expansion                      # |
   @ ./REPL[44]:7 [inlined]               # ⌋
 [3] f(x::Int64)
   @ Main ./REPL[45]:6
 [4] top-level scope
   @ REPL[46]:1

Does anyone know how to remove lines from the stacktrace before throwing?

[jameson]

You could instead do

throw(@outline DomainError(lazy"f requires inputs greater than 0, you gave $x which is less than 0"))

where we define

macro outline(x)
    return esc(:((() -> $x)()))
end

[Mason]

Hm, unfortunately, that seems to generate a bunch more inlined code, and is slower.

julia> macro outline(x)
           return esc(:((() -> $x)()))
       end
@outline (macro with 1 method)

julia> function f(x)
           if x >= 0
               x + 1
           else
               #throw(DomainError("f requires inputs greater than 0, you gave $x which is less than 0"))
               throw(@outline DomainError(lazy"f requires inputs greater than 0, you gave $x which is less than 0"))
           end
       end;

julia> @btime f($(Ref(1.0))[])
  2.809 ns (0 allocations: 0 bytes)
2.0

julia> code_llvm(f, Tuple{Int})
;  @ REPL[24]:1 within `f`
define i64 @julia_f_624(i64 signext %0) #0 {
top:
  %gcframe16 = alloca [5 x {}*], align 16
  %gcframe16.sub = getelementptr inbounds [5 x {}*], [5 x {}*]* %gcframe16, i64 0, i64 0
  %1 = bitcast [5 x {}*]* %gcframe16 to i8*
  call void @llvm.memset.p0i8.i32(i8* noundef nonnull align 16 dereferenceable(40) %1, i8 0, i32 40, i1 false)
  %2 = getelementptr inbounds [5 x {}*], [5 x {}*]* %gcframe16, i64 0, i64 2
  %thread_ptr = call i8* asm "movq %fs:0, $0", "=r"() #6
  %ppgcstack_i8 = getelementptr i8, i8* %thread_ptr, i64 -8
  %ppgcstack = bitcast i8* %ppgcstack_i8 to {}****
  %pgcstack = load {}***, {}**** %ppgcstack, align 8
;  @ REPL[24]:2 within `f`
; ┌ @ operators.jl:429 within `>=`
; │┌ @ int.jl:481 within `<=`
    %3 = bitcast [5 x {}*]* %gcframe16 to i64*
    store i64 12, i64* %3, align 16
    %4 = getelementptr inbounds [5 x {}*], [5 x {}*]* %gcframe16, i64 0, i64 1
    %5 = bitcast {}** %4 to {}***
    %6 = load {}**, {}*** %pgcstack, align 8
    store {}** %6, {}*** %5, align 8
    %7 = bitcast {}*** %pgcstack to {}***
    store {}** %gcframe16.sub, {}*** %7, align 8
    %8 = icmp slt i64 %0, 0
; └└
  br i1 %8, label %L5, label %L3

L3:                                               ; preds = %top
;  @ REPL[24]:3 within `f`
; ┌ @ int.jl:87 within `+`
   %9 = add nuw i64 %0, 1
   %10 = load {}*, {}** %4, align 8
   %11 = bitcast {}*** %pgcstack to {}**
   store {}* %10, {}** %11, align 8
; └
  ret i64 %9

L5:                                               ; preds = %top
;  @ REPL[24]:6 within `f`
; ┌ @ REPL[23]:2 within `#13`
; │┌ @ strings/lazy.jl:19 within `LazyString`
    %ptls_field17 = getelementptr inbounds {}**, {}*** %pgcstack, i64 2
    %12 = bitcast {}*** %ptls_field17 to i8**
    %ptls_load1819 = load i8*, i8** %12, align 8
    %13 = call noalias nonnull {}* @ijl_gc_pool_alloc(i8* %ptls_load1819, i32 1440, i32 32) #7
    %14 = bitcast {}* %13 to i64*
    %15 = getelementptr inbounds i64, i64* %14, i64 -1
    store atomic i64 139939984728400, i64* %15 unordered, align 8
    %16 = bitcast {}* %13 to {}**
    %17 = bitcast {}* %13 to <2 x {}*>*
    store <2 x {}*> zeroinitializer, <2 x {}*>* %17, align 8
    %18 = getelementptr inbounds [5 x {}*], [5 x {}*]* %gcframe16, i64 0, i64 4
    store {}* %13, {}** %18, align 16
    %ptls_load132021 = load i8*, i8** %12, align 8
    %19 = call noalias nonnull {}* @ijl_gc_pool_alloc(i8* %ptls_load132021, i32 1440, i32 32) #7
    %20 = bitcast {}* %19 to i64*
    %21 = getelementptr inbounds i64, i64* %20, i64 -1
    store atomic i64 139939991032544, i64* %21 unordered, align 8
    %22 = bitcast {}* %19 to { {}*, i64, {}* }*
    %.repack = bitcast {}* %19 to {}**
    store {}* inttoptr (i64 139940196512528 to {}*), {}** %.repack, align 8
    %.repack6 = getelementptr inbounds { {}*, i64, {}* }, { {}*, i64, {}* }* %22, i64 0, i32 1
    store i64 %0, i64* %.repack6, align 8
    %.repack8 = getelementptr inbounds { {}*, i64, {}* }, { {}*, i64, {}* }* %22, i64 0, i32 2
    store {}* inttoptr (i64 139937774046432 to {}*), {}** %.repack8, align 8
    store {}* %19, {}** %16, align 8
    %23 = load atomic i64, i64* %15 unordered, align 8
    %24 = and i64 %23, 3
    %25 = icmp eq i64 %24, 3
    br i1 %25, label %26, label %27

26:                                               ; preds = %L5
    call void @ijl_gc_queue_root({}* nonnull %13)
    br label %27

27:                                               ; preds = %26, %L5
    %28 = bitcast {}** %2 to [2 x {}*]*
; │└
   call void @j_DomainError_626([2 x {}*]* noalias nocapture nonnull sret([2 x {}*]) %28, {}* nonnull readonly %13) #0
; └
  %ptls_load152223 = load i8*, i8** %12, align 8
  %29 = call noalias nonnull {}* @ijl_gc_pool_alloc(i8* %ptls_load152223, i32 1440, i32 32) #7
  %30 = bitcast {}* %29 to i64*
  %31 = getelementptr inbounds i64, i64* %30, i64 -1
  store atomic i64 139939981380608, i64* %31 unordered, align 8
  %32 = bitcast {}* %29 to i8*
  %33 = bitcast {}** %2 to i8*
  call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 8 dereferenceable(16) %32, i8* noundef nonnull align 16 dereferenceable(16) %33, i64 16, i1 false)
  call void @ijl_throw({}* %29)
  unreachable
}

Marking the function inside @outline as @noinline and avoiding capturing variables didn’t seem to help.

[jar1]

Python recently introduced some nice new error messages.

In order to identify the range of source code being executed when exceptions are raised, this proposal requires adding new data for every bytecode instruction.
…
We understand that the extra cost of this information may not be acceptable for some users, so we propose an opt-out mechanism which will cause generated code objects to not have the extra information while also allowing pyc files to not include the extra information.

PEP 657 – Include Fine Grained Error Locations in Tracebacks | peps.python.org