Make Julia’s Error Codes Even Better Than Elm’s

Internals & Design
22

Thank you to you and to @StefanKarpinski for sharing this! I’ll give it a try :smiley:

1 Like
23

In Elm, array indexing returns Just val or Nothing.

24

Ah, so out of bounds indexing returns nothing?

25

That’s how it looks to me:

https://package.elm-lang.org/packages/elm/core/latest/Array#get

26

https://guide.elm-lang.org/error_handling/
Looks similar to Rust where it makes you handle the error.

27

Printing the offending value is useful when it’s a simple value with a short representation. But if it’s not done carefully, you may end up printing a very long representation of some value. This is a problem because:

  • printing the error might be very slow, especially in a scrolling terminal such as inside vscode
  • it may fill up a terminal so you can’t see what came before the error
  • in a notebook environment you get a huge wall of text to scroll past, which is just annoying.

In my experience, this is a common problem in Julia packages, and I am more frequently annoyed by error messages that are too long than error messages that are missing information.

5 Likes
28

For example in this case,

julia> f(x,y) = sqrt(x) < sqrt(y)
f (generic function with 1 method)

julia> f(5,-1)
ERROR: DomainError with -1.0:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
Stacktrace:
 [1] throw_complex_domainerror(f::Symbol, x::Float64)
   @ Base.Math ./math.jl:33
 [2] sqrt
   @ ./math.jl:567 [inlined]
 [3] sqrt
   @ ./math.jl:1221 [inlined]
 [4] f(x::Int64, y::Int64)
   @ Main ./REPL[2]:1
 [5] top-level scope
   @ REPL[3]:1

the message could specify the location of the problem more precisely.

f(x,y) = sqrt(x) < sqrt(y)
                   ^^^^^^^

This would be helpful beacuse it is currently not clear which argument is the problem.

2 Likes
29

Good luck with this if the function doesn’t exist “physically” in your source code, but comes from macro expansions or eval calls. :grinning:. (I do encounter such situations occasionally.)

3 Likes
30

JuliaSyntax.jl will help a lot in most of the situations except with macro-generated code. I think error messages from that kind of code will still be ugly.

2 Likes
31

Stefan Karpinski already explained why this is difficult to implement. The additional problem is that “specify the location of the problem more precisely” doesn’t make sense as a requirement.
Consider that the error happens at the top of the call stack, and what you want is basically in-depth info on some stack trace entry. It may be possible (although difficult) to implement in-depth info for each stack trace entry, but I don’t think it’s possible for Julia to guess what entry you want in-depth info for.

I think this might be doable as an interface in a GUI tool, though, just let the user select any entry and expand it.

3 Likes
32

I get that this is prettier and easier to understand, but the line number and column in the stack trace isn’t enough to identify where the error is? If you’re using vs-code for example, just CTRL + click the line number and column in the stack trace and the editor will bring you to that specific line of code.

The only thing i don’t like about julia’s errors is the order of the stack trace. Most of the time i have to scroll up to see where the error came from.

1 Like
33

The second invocation of sqrt is the problem but that column isn’t given anywhere.

f(x,y) = sqrt(x) < sqrt(y)

f(1,-1)

ERROR: LoadError: DomainError with -1.0:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
Stacktrace:
 [1] throw_complex_domainerror(f::Symbol, x::Float64)
   @ Base.Math ./math.jl:33
 [2] sqrt
   @ ./math.jl:591 [inlined]
 [3] sqrt
   @ ./math.jl:1372 [inlined]
 [4] f(x::Int64, y::Int64)
   @ Main /tmp/foo.jl:1
 [5] top-level scope
   @ /tmp/foo.jl:3
in expression starting at /tmp/foo.jl:3
34

I’m not an expert, but I think JuliaSyntax will only help with improving parsing error messages. I don’t think that kind of syntactical information will be carried around by JIT compiled functions.

1 Like
35

Yes, check out the huge message by Stefan Karpinski above. It explains why this is very difficult to implement currently, and would probably require improving LLVM first.

1 Like
36

Yes, you’re right, my mistake. julia only gives the line number.

37

Errors don’t usually come from LLVM. The thing that would be missing is lowering. We lose a lot of information when we lower Julia code to our IR which hopefully could be solved by moving lowering to Julia as well.
Unfortunately lowering is a bit more involved than parsing

38

It’s really really hard to beat Elm’s error report. Elm (and Haskell) are perfect platforms of dedicated compile time error because they have a relative simple type system (basically Hindley–Milner type system . This system enjoys many great properties. That’s why you can not only find the errors, but also give useful suggestions to correct them, and all of these can be done in a fast manner. C/Go language has an even simple type system, so it’s not surprising that they both compile quite fast. Quote from the article:

Technical Note: I found that generating such specific error messages required no significant changes to the type inference algorithm and imposed no noticeable performance cost.

What’s more, Julia does not have a well-defined concept of compile time error (specifically, type error). Type stability or type privacy are problematic as they are vague terms for “program errors”, so not every one will agree on a single definition of these terms in Julia, so you can’t just raise error.

Previously, I tried to design a simple type checker for Julia, to capture type errors and improve code qualities (so you can get better error reports than manually calling Cthulhu or code_typed). Unfortunately, it doesn’t work as expected and the output is quite disappointing. As a lot of information is lost on lowered form IR, so you cannot locate the expression precisely (you have only line number, which is imprecise). Additionally, people always perform type calculations in an implicit way (for example, they use isnothing to narrow type of variable). So if you use a simple type inference algorithm, let’s say that you directly work on AST, you may misjudge some programs as ill-typed.

Another advantage of Haskell/Elm is that their programs are more type-annotated than Julia. So if you have some errors, you can isolate the erroneous part easily. In contrast, Julia has more floating parts and types of programs can depend on each others. To isolate the real cause of the error, you need to look around the environment and maybe produce more error messages (the situation here is much like that in C++. C++ also suffers from verbose errors caused by templates).

6 Likes
39

In fact, Julia needs better error messages. Any error inside a dynamics function of DifferetialEquations.jl leads to a huge text which is not relevant 99% of time, since the error normally is something in a function outside the DifferentialEquations.jl packages.

using DifferentialEquations

f(u,p,t) = 1.01*up
u0 = 1/2
tspan = (0.0,1.0)
prob = ODEProblem(f,u0,tspan)
sol = solve(prob, Tsit5(), reltol=1e-8, abstol=1e-8)

julia> include("od.jl")
ERROR: LoadError: UndefVarError: up not defined
Stacktrace:
  [1] f(u::Float64, p::SciMLBase.NullParameters, t::Float64)
    @ Main ~/tmp/od.jl:4
  [2] ODEFunction
    @ ~/.julia/packages/SciMLBase/xWByK/src/scimlfunctions.jl:1962 [inlined]
  [3] initialize!(integrator::OrdinaryDiffEq.ODEIntegrator{Tsit5{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}, false, Float64, Nothing, Float64, SciMLBase.NullParameters, Float64, Float64, Float64, Float64, Vector{Float64}, ODESolution{Float64, 1, Vector{Float64}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Float64}}, ODEProblem{Float64, Tuple{Float64, Float64}, false, SciMLBase.NullParameters, ODEFunction{false, SciMLBase.AutoSpecialize, typeof(f), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing, Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, Tsit5{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}, OrdinaryDiffEq.InterpolationData{ODEFunction{false, SciMLBase.AutoSpecialize, typeof(f), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing, Nothing}, Vector{Float64}, Vector{Float64}, Vector{Vector{Float64}}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}}, DiffEqBase.DEStats}, ODEFunction{false, SciMLBase.AutoSpecialize, typeof(f), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing, Nothing}, OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64}, OrdinaryDiffEq.DEOptions{Float64, Float64, Float64, Float64, PIController{Rational{Int64}}, typeof(DiffEqBase.ODE_DEFAULT_NORM), typeof(LinearAlgebra.opnorm), Nothing, CallbackSet{Tuple{}, Tuple{}}, typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), DataStructures.BinaryHeap{Float64, DataStructures.FasterForward}, DataStructures.BinaryHeap{Float64, DataStructures.FasterForward}, Nothing, Nothing, Int64, Tuple{}, Tuple{}, Tuple{}}, Float64, Float64, Nothing, OrdinaryDiffEq.DefaultInit}, cache::OrdinaryDiffEq.Tsit5ConstantCache{Float64, Float64})
    @ OrdinaryDiffEq ~/.julia/packages/OrdinaryDiffEq/vfMzV/src/perform_step/low_order_rk_perform_step.jl:672
  [4] __init(prob::ODEProblem{Float64, Tuple{Float64, Float64}, false, SciMLBase.NullParameters, ODEFunction{false, SciMLBase.AutoSpecialize, typeof(f), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing, Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, alg::Tsit5{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}, timeseries_init::Tuple{}, ts_init::Tuple{}, ks_init::Tuple{}, recompile::Type{Val{true}}; saveat::Tuple{}, tstops::Tuple{}, d_discontinuities::Tuple{}, save_idxs::Nothing, save_everystep::Bool, save_on::Bool, save_start::Bool, save_end::Nothing, callback::Nothing, dense::Bool, calck::Bool, dt::Float64, dtmin::Nothing, dtmax::Float64, force_dtmin::Bool, adaptive::Bool, gamma::Rational{Int64}, abstol::Float64, reltol::Float64, qmin::Rational{Int64}, qmax::Int64, qsteady_min::Int64, qsteady_max::Int64, beta1::Nothing, beta2::Nothing, qoldinit::Rational{Int64}, controller::Nothing, fullnormalize::Bool, failfactor::Int64, maxiters::Int64, internalnorm::typeof(DiffEqBase.ODE_DEFAULT_NORM), internalopnorm::typeof(LinearAlgebra.opnorm), isoutofdomain::typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), unstable_check::typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), verbose::Bool, timeseries_errors::Bool, dense_errors::Bool, advance_to_tstop::Bool, stop_at_next_tstop::Bool, initialize_save::Bool, progress::Bool, progress_steps::Int64, progress_name::String, progress_message::typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), userdata::Nothing, allow_extrapolation::Bool, initialize_integrator::Bool, alias_u0::Bool, alias_du0::Bool, initializealg::OrdinaryDiffEq.DefaultInit, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ OrdinaryDiffEq ~/.julia/packages/OrdinaryDiffEq/vfMzV/src/solve.jl:493
  [5] #__solve#562
    @ ~/.julia/packages/OrdinaryDiffEq/vfMzV/src/solve.jl:5 [inlined]
  [6] #solve_call#26
    @ ~/.julia/packages/DiffEqBase/5rKYk/src/solve.jl:472 [inlined]
  [7] solve_up(prob::ODEProblem{Float64, Tuple{Float64, Float64}, false, SciMLBase.NullParameters, ODEFunction{false, SciMLBase.AutoSpecialize, typeof(f), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing, Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, sensealg::Nothing, u0::Float64, p::SciMLBase.NullParameters, args::Tsit5{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}; kwargs::Base.Pairs{Symbol, Float64, Tuple{Symbol, Symbol}, NamedTuple{(:reltol, :abstol), Tuple{Float64, Float64}}})
    @ DiffEqBase ~/.julia/packages/DiffEqBase/5rKYk/src/solve.jl:834
  [8] #solve#31
    @ ~/.julia/packages/DiffEqBase/5rKYk/src/solve.jl:801 [inlined]
  [9] top-level scope
    @ ~/tmp/od.jl:8
 [10] include(fname::String)
    @ Base.MainInclude ./client.jl:476
 [11] top-level scope
    @ REPL[3]:1
in expression starting at /Users/ronan.arraes/tmp/od.jl:8
2 Likes
40

Have you seen this?

I was surprised how well it works.

Tomas

10 Likes
41

Actually no! Thanks!