Use exceptions vs other patterns?

General Usage
1

Again, I don’t understand what you mean here.

I read elsewhere in the Julia documentation that

Exception handling in Julia is done using try — catch — finally, instead of try — except — finally. In contrast to Python, it is not recommended to use exception handling as part of the normal workflow in Julia (compared with Python, Julia is faster at ordinary control flow but slower at exception-catching).

I’m interested to learn more about this.

  • If Julia doesn’t recommend the use of exceptions for handling error state which occurs in program flow, then what is the recommended method?
  • I am aware that sometimes nothing or missing might be the most appropriate return value, but this should only be used when it is syntactically sensible and meaningful to do so. Put another way, you shouldn’t default to using nothing to indicate error state, because nothing means “there is no value”, it does not mean “the state is an error state”.
  • Hence I am slightly confused. Julia doesn’t have a result type (or does it?), like Rust does. The only other ways of handling error state which I am aware of from other languages are:
  1. Returning integer value error codes, which is what C does (this is not a good idea) and the software engineering world agreed that it is not a good idea a long time ago. (Let me know if you want an explanation for this?)
  2. Raising exceptions, which is what most other languages do

Can anyone share their experience relating to this?

2

Exceptions are arguably not a good choice for control flow in any language. As their name says. See, e.g.:

In Julia using exceptions for control flow is more often a bad idea because it results in worse performance, as exception handling is difficult to compile into efficient code. That said, AFAIK, there’s still low-hanging fruit for optimizing exception handling in the compiler.

Well, define a new type if you need one.

Depends on what you mean by “have”. It’s possible to define such a type, and there’s a myriad of sum type packages making that easier.

3

I see your point. What I meant was not part of the core language or standard library.

Since writing this I did find a package which provides Option and Result types.

Results · Julia Packages

4

FWIW, I don’t recommend using that specific package, it doesn’t define a named type for Result, rather it’s just a type Union. Giving rise to type instability. Maybe take a look at one of these instead:

SumTypes.jl

LightSumTypes.jl

Moshi.jl

5

I missed the link you posted. FYI from the same site:

What are the best practices when implementing C++ error handling? - Software Engineering Stack Exchange

Exceptions vs [something else] is one of those things in software which is a bit fashionable and changes with the times.

In the beginning there was C, and arbitrary error codes were used to indicate error cases. There are a number of problems with this, some of them described in that post linked above.

Exceptions became the de-facto standard with the rise of C++ and Java. They have a number of advantages of just using error codes, but there are also some disadvantages. (The primary one being the caller doesn’t have to handle them, which often results in exceptions not being caught where they should and thus propagating further up the stack.)

Arguably a return type which is a union of some error type and some other return value type is a better design. Rust uses that, but this is part of the core language. This design also has its disadvantages. Primarily a proliferation of many possible combination of types. In Rust lots of people get around this by dispatching the error part of the Result type union dynamically. It significantly reduces the amount of typing required to implement something, but it’s typically expected to be slightly slower. You still have to define your own type to implement this, though.

6

I agree that nothing doesn’t have any information about the failure in the way that an exception would. Some(x) | nothing works when that information isn’t needed. When error info is needed, Julia should have a better option. I’m hoping it’s built on Moshi or similar, rather than on exceptions.

7

Perhaps a future update could just copy what Rust has, where a Result is either an Ok(T) or an Error(E) where T is a type, and E is a type used to represent the error.

I’m new to Julia so I have no idea if this would be a good/bad design in relation to performance implications. Given that all data would be on the stack, I would assume performance would be quite good.

8

Why? Packages solving this were already mentioned above.

9

Well, if exceptions are part of the core language runtime, and if exceptions are discouraged from being used, should not any alternative be part of the core language too?

10

It’s not that exceptions are discouraged, it’s that exceptions are discouraged in the parts of your code that are performance critical.

1 Like
11

I think exceptions should often be discouraged, not only for performance but because failures are just normal state and it’s good to represent them as normal values. Using sum types like Ok{T} | Err{E} is more verbose but it’s also more local, more visible, and more controlled. Failure values can help you understand and do what you mean to do.

Moshi is still new so I don’t know if that’ll end up being the right basis, but I do think we should have an error type and Base should be rewritten with it in mind.

1 Like
12

To clarify for @edward-quant, the alternative to exceptions that Base Julia (and most of the ecosystem) uses when performance is critical is Union{T, Nothing}. For example, findfirst(x) returns a Union{keytype(x), Nothing}. This approach works fine for many simple functions. There are advantages and disadvantages to this approach:

Advantage

You can write quick and dirty code that works even though you haven’t considered nothing return values, like this:

julia> x = [false, true];

julia> 10 + findfirst(x)
12

Disadvantage

The quick and dirty code that you wrote might have bugs in it because you didn’t handle the nothing cases:

julia> x = [false, false];

julia> 10 + findfirst(x)
ERROR: MethodError: no method matching +(::Int64, ::Nothing)

Discussion

Given that Julia is a dynamic, interactive language, the ability to run quick and dirty code in the REPL is an advantage that should not be quickly dismissed. If findfirst returned an Ok{T} | Err{E}, then you wouldn’t be able to run 10 + findfirst(x) in the REPL—it would throw a MethodError for the addition of 10 and Ok(2). The Ok{T} | Err{E} result type shines more in a static, compiled language.

2 Likes
13

If you’re interested in static type-checking, JET can already detect method errors due to not accounting for nothing return values. No need for a Result type! :slight_smile:

julia> using JET

julia> foo(x) = 10 + findfirst(x)
foo (generic function with 1 method)

julia> @report_call foo([false, true])
═════ 1 possible error found ═════
┌ foo(x::Vector{Bool}) @ Main ./REPL[12]:1
│ no matching method found `+(::Int64, ::Nothing)` (1/2 union split): (10 + findfirst(x::Vector{Bool})::Union{Nothing, Int64})
3 Likes
14

This only makes sense in some contexts. In this context it makes sense, because the result of not finding something should not be represented by an error but should be represented by returning a value which represents the state “not found” or “found nothing” - which this does.

It does not make sense in general - error states should not be represented by returning a “nothing value”. In this particular case you do not have an error, however if the find algorithm were to somehow enter an error state, then an error value should be raised or returned, it should not also return “nothing value” in such cases. (For all the common finding algorithms I know, I struggle to imagine such a case where an error state would be reached - but this is besides the point.)

Sorry if this comes across as excessively pedantic - I thought it might be useful to raise these points in detail.

1 Like
15

findfirst returning Union{T, Nothing} is ambiguous and cannot be used safely in generic programming where there is no proof ruling out Nothing <: T, i.e. in a library function where the caller controls T.

Returning Union{Some{T}, Nothing} is at least unambiguous, but doesn’t communicate any information about the failure other than that one bit – no type, location, value. And there’s no API for safely using it: something(x::T) returns x for any T !== Nothing not just T <: Some, which destroys information. Let alone an API for conveniently using it such as map, and_then, etc. And it’s type-unstable (which can be helpful for dispatch, but is suboptimal for inference).

1 Like
16

The “error state” is not a well defined term as far as I know. Who’s to say that “not found” is not an “error state”?

Anyways, the point is that Julia is a dynamic language and normally uses Union return types, not sum types. If you need to return an error value with more information than nothing, you can return any other type you like. If a function has more than one “error state”, you can return a larger union, like Union{TheNonErrorReturnType, MyErrorType1, MyErrorType2}.

I agree that Some{T} is needed for some functions, but note that Union{Some{T}, Nothing} is still a union type, not a sum type. The Some{T} type is only needed for functions where nothing is one of the possible “non-error values”. But there are plenty of functions where that is not an issue. To take an artificial, toy example:

using Distributions

struct TooSmall end
struct TooBig end

function my_quantile(d::Distribution{Univariate, Continuous}, q::Real)
    if  q < 0
        TooSmall()
    elseif q > 1
        TooBig()
    else
        quantile(d, q)
    end
end

The return type is Union{Real, TooSmall, TooBig}. There’s no need for a Some here, because the “non-error” return value will always be of type Real—a “non-error” return value will never have type TooSmall or TooBig.

The larger question here is whether to use union types or sum types for return values. And as I’ve already mentioned, there are downsides to using sum types. The following talk by Rich Hickey about union vs sum types for return values might be of interest:

1 Like
17

It is well defined. To explain, it is not an error for an item to not exist inside of a container. It is not an error for a dictionary to not contain the value “hello” as a key.

Similarly, it is not an error for a vector to not have the value 2 in it. If it was, the overwhelming majority of vectors which were constructed would all be in an error state, because (presumably) most vectors do not contain 2. Since 2 is not special, this would imply that any vector which does not contain the set of all possible values is in an error state. Such a proposal is nonsensical, because it would imply that all “non-error-state” vectors are all the same, contain the same values, and don’t fit into the memory of most (all?) computers which currently exist. (2^64 is a large number. That is the number of possible values, if each value is 64 bits long.)

The code which uses the dictionary API might consider it to be an error if a key is missing, but the “errorness” of that relates to the client which uses the dictionary API, it is not internal to the dictionary itself.

This is not good software design for several reasons. Conceptually, an operation should succeed with a value or error with an error value. The error value itself might be an aggregation of several possible options, but conceptually the state is either an error or a success. Not an error or an error or a success.

A better design would be Union{Real, QuartileError} where QuartileError contains the information about why this is an error.

Actually, you already have an error for this exact case. It’s a domain error.

I cant’ speak to the exact internal design which Julia should use, if it were to try and move from the existing state of the world where exceptions are used to signal something has gone wrong to a world where Rust Result style return values were used.

Much of this wouldn’t make much sense without a destructuring match construct. (Again, something Rust has.) I don’t know if Julia has this or not. I’m aware it has a match statement but I’m not sure if that does the same thing (yet).

18

I think there is a way of thinking of it where you only use the throwing functions when your correctness criteria assume that the value is present in the collection, and the exception is only there as an extra runtime safety check. That way the programmer doesn’t have to fill up their screen with code handling states that they’re happy to assume never occur.

But for those other cases where we’re not able to assume them away, I agree it’s best to have informative failure values, type-stable exclusive pattern matching, etc.

19

There’s not much practical difference between this,

Union{OutType, Err{Union{Err1, Err2}}}

and this,

Union{OutType, Err1, Err2}

Both are perfectly serviceable approaches.

Actually, in Julia the Union{Real, TooSmall, TooBig} return type can be more convenient, since callers can dispatch on the type of the return value from my_quantile, like this:

foo(x::Real) = 2x
foo(::TooSmall) = println("hello")
foo(::TooBig) = println("world")

foo(my_quantile(a)) # Return a number, print "hello", or print "world".

I guess you missed the part where I said “artificial, toy example”. Let’s make it even more artificial:

struct Foo end
struct Bar end

function asdf(x)
    x isa String || return Foo()
    x == "a" ? "hello" : Bar()
end

In this case the return type is Union{String, Foo, Bar}.

Anyhow, that’s all I have to say on this subject. Welcome to the Julia language. I hope you enjoy the language, and try to not jump to conclusions about the language based on your experiences with other languages. :slight_smile:

20

There is a huge difference between these two things. They have completely different semantics. Software engineering is not about “having the right data in a structure” it is about conveying a precise meaning as to what a code is intended to do.

I appreciate if you have never worked for a software house with a team who had a focus on writing clear and meaningful code (not all teams do) then what I am saying may appear esotetic, but I can assure you this is not a triviality.

I used to work in a scientific field, and the software which was written there was considered good enough if it did the correct thing. Meaning some data went in and some data came out with the right values. Good software is more than just that. It is maintainable by large teams (meaning it is clear to understand when reading it) and it is flexible (meaning it can be changed without huge refactoring jobs).

If you’re interested in learning more about this, the Clean Code book is a good place to start. It is considered a bit of a “meme” book in the SWE world. Not all of the advice in it is good advice, it has to be said. But enough of it is to be considered worth reading. You could also read the GOF Design Patterns - but that’s not relevant to Julia. It’s for OOP languages with encapsulation like Java and C++.

I would guess you’re coming from the science world, given what you said.

It’s often a good idea to listen to advice from people with experience in a broad range of domains. I didn’t join this thread for egotistical reasons, I just wanted to add support for the initial proposal made by OP.