Julia crashes when redifining a method of one(T::Type)

Internals & Design
1

I found that the following code makes Julia crash in macOS Sierra (10.12.6):

import Base: one
one(::Type{<:Real}) = nothing

This happens also with when replacing nothing by, for example, Real.

The output I get is:

Error showing value of type typeof(one):
SYSTEM: show(lasterr) caused an error
MethodError(Int64, (nothing,), 0x00000000000061d6)

Stacktrace:
[1] oneunit(::Type{Int64}) at ./number.jl:321
[2] first(::Base.OneTo{Int64}) at ./range.jl:544
[3] first(::LinearIndices{1,Tuple{Base.OneTo{Int64}}}) at ./indices.jl:422
[4] methods(::Any, ::Any) at ./reflection.jl:754
[5] methods(::Any) at ./reflection.jl:769
[6] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol(“text/plain”)}, ::Function) at ./show.jl:34
[7] display(::REPL.REPLDisplay{REPL.LineEditREPL}, ::MIME{Symbol(“text/plain”)}, ::Function) at /Users/userX/.julia/packages/OhMyREPL/f919j/src/output_prompt_overwrite.jl:6
[8] display(::REPL.REPLDisplay, ::Any) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:135
[9] display(::Any) at ./multimedia.jl:287
[10] #invokelatest#1 at ./essentials.jl:697 [inlined]
[11] invokelatest at ./essentials.jl:696 [inlined]
[12] print_response(::IO, ::Any, ::Any, ::Bool, ::Bool, ::Any) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:154
[13] print_response(::REPL.AbstractREPL, ::Any, ::Any, ::Bool, ::Bool) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:139
[14] (::getfield(REPL, Symbol(“#do_respond#40”)){Bool,getfield(REPL, Symbol(“##50#59”)){REPL.LineEditREPL,REPL.REPLHistoryProvider},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:708
[15] #invokelatest#1 at ./essentials.jl:697 [inlined]
[16] invokelatest at ./essentials.jl:696 [inlined]
[17] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:2261
[18] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:1029
[19] run_repl(::REPL.AbstractREPL, ::Any) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:191
[20] (::getfield(Base, Symbol(“##719#721”)){Bool,Bool,Bool,Bool})(::Module) at ./logging.jl:311
[21] #invokelatest#1 at ./essentials.jl:697 [inlined]
[22] invokelatest at ./essentials.jl:696 [inlined]
[23] macro expansion at ./logging.jl:308 [inlined]
[24] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:330
[25] exec_options(::Base.JLOptions) at ./client.jl:242
[26] start() at ./client.jl:421
julia>
fatal: error thrown and no exception handler available.
MethodError(f=Int64, args=(nothing,), world=0x00000000000061d6)
rec_backtrace at /Users/osx/buildbot/slave/package_osx64/build/src/stackwalk.c:94
record_backtrace at /Users/osx/buildbot/slave/package_osx64/build/src/task.c:246 [inlined]
jl_throw at /Users/osx/buildbot/slave/package_osx64/build/src/task.c:577
jl_method_error_bare at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1616
jl_method_error at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1634
jl_lookup_generic at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:2161
jl_apply_generic at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:2181
oneunit at ./number.jl:321
first at ./range.jl:544
#83 at ./abstractarray.jl:85
_tuple_any at ./tuple.jl:415
unknown function (ip: 0x12c31acb7)
_tuple_any at ./tuple.jl:412
unknown function (ip: 0x12c31ac09)
has_offset_axes at ./abstractarray.jl:85
Type at ./iobuffer.jl:19
Type at ./iobuffer.jl:27
#IOBuffer#299 at ./iobuffer.jl:98
unknown function (ip: 0x12c31a8bc)
Type at ./none:0
#IOBuffer#300 at ./iobuffer.jl:114
unknown function (ip: 0x12c31a67c)
Type at ./iobuffer.jl:112
#with_output_color#660 at ./util.jl:364
unknown function (ip: 0x12c31f86b)
jl_fptr_trampoline at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1831
#with_output_color at ./none:0
unknown function (ip: 0x12c31f800)
jl_fptr_trampoline at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1831
#printstyled#661 at ./util.jl:398
unknown function (ip: 0x12c31f717)
jl_fptr_trampoline at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1831
#printstyled at ./none:0
jl_fptr_trampoline at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1831
display_error at ./client.jl:93
jl_fptr_trampoline at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1831
display_error at ./client.jl:102
jl_fptr_trampoline at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1831
jl_apply at /Users/osx/buildbot/slave/package_osx64/build/src/./julia.h:1537 [inlined]
jl_f__apply at /Users/osx/buildbot/slave/package_osx64/build/src/builtins.c:556
jl_f__apply_latest at /Users/osx/buildbot/slave/package_osx64/build/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:697 [inlined]
invokelatest at ./essentials.jl:696 [inlined]
_start at ./client.jl:423
true_main at /usr/local/bin/julia (unknown line)
main at /usr/local/bin/julia (unknown line)

2

Redefining Base-methods is a no-no and can lead to all sorts of behavior, including crashing Julia. This makes sense, as one and other methods in Base are not just defined there but also used for core functionality. Thus redefining them will break that core functionality and Julia with it.

2 Likes
3

I am curious

  1. why you are doing this,
  2. what you expected would happen.
4

This might work if you don’t commit type piracy. It crashes because the type piracy destroys the core functionality. But you should be able to extend one for your own types. I’ve done this in the Reduce package.

5

If you don’t do type-piracy it should work fine. In fact, if it doesn’t then it is indeed a bug.

1 Like
6

There have been other examples lately where fiddling with core functionality causes Julia to crash.

But being able to crash the system so easily creates a bad impression of Julia.
“Crash” = “unstable system” = “Julia is an immature programming language”.

How about raising an error instead of crashing? For example
“I’m sorry Dave. I’m afraid I can’t do that.”

7

I know everyone is going to think that I’m a crazy weirdo, but from the moment I first discovered Julia I always thought that the fact that this is possible is just so cool. As a fun exercise, start your REPL and do Base.:+(a,b) = nothing and see how far you get (the answer is: surprisingly far).

Of course you can probably infer that my vote would be for continuing not to throw an error when such methods are re-defined. Other than my giddy excitement about how comprehensively I am able to screw up a run-time, my other argument is that it might be very useful to be able to do this in isolation for testing. In fact, typically if I do any work on the Julia repo itself I test my changes using type piracy.

4 Likes
8

Of course, it makes perfect sense now that you say it.

Thanks for your help.

9

This has nothing to do with the maturity of the language. It is deliberate due to the design choices.

That could be one way of handling it, but then how do you decide in general whether it can be allowed or not?

Since Julia does not have these kind of limitations that stop you from crashing Julia, you can extend Julia in ways that you might otherwise not be able to, provided the effort is spent to be stable and not break things.

Typically, it is a bad idea to do type piracy, especially on the core functionality of Julia itself, but because of the flexibility of Julia, with wisdom you can usually achieve a design with stable dispatch that doesn’t crash Julia.

This is the same reason that Julia has unsafe integer overflows by default. It is a design choice.

10

I was just tinkering with the Type system to define a broader and extendable set of scalar (i.e. unitless) numbers.

My idea was to define a module that would serve as a base and would provide a basic infrastructure for any other member of the set. Here a very simplified version of the code:

module MyNumber

const operations = (:+, :-, :*, :/) # :one removed --> Julia crashes
for op in operations
    @eval import Base: $op
    @eval export $op
end

export MyAbstractNumberType, MyNumberType1, MyNumberTypes

abstract type MyAbstractNumberType end

struct MyNumberType1 <: MyAbstractNumberType end
const mynumbertype1 = MyNumberType1()

MyNumberTypes = Union{MyNumberType1,Real}

for op in operations
    @eval $op(::Type{<:MyNumberTypes}) = MyNumberType
end

# More methods of the kind f(x::MyNumberTypes)

end

Thinking that another module could look like:

module MyNumber2

using MyNumber
import MyNumber: MyNumberTypes
export MyNumberTypes

struct MyNumberType2 <: MyAbstractNumberType end
const mynumbertype2 = MyNumberType2()

MyNumberTypes = Union{MyNumberTypes, MyNumberType2}

# No need to define new methods because they have been defined in module MyNumber

end

Then I realized that Julia was crashing when trying to compile and I managed to isolate the failure to trying to redefine one(::Type{<:Real}).

I removed then Real from MyNumberTypes and I could go ahead, but then I realized that trying to modify MyNumberTypes (defined in module MyNumber) from module MyNumber2 is not allowed. I have to do something like:

module MyNumber2

using MyNumber
export MyNumberTypes

struct MyNumberType2 <: MyAbstractNumberType end
const mynumbertype2 = MyNumberType2()

MyNumberTypes = Union{MyNumber.MyNumberTypes, MyNumberType2}

# No need to define new methods because they have been defined in module MyNumber

end
11

I don’t understand why you say this. I imagine I could slash the tires of my bike rather easily with sharp enough object. That does not mean it isn’t a good bike. It simply means that I should not slash the tires if I want to keep using it.

12

Also, something that is igored every time this comes up. You did NOT crash julia. All the internal core systems are working perfectly fine. You did not get a compiler error, you got a REPL error. Since the REPL here is intended to response to everything you define, this is also the expected behavior.

In short, you write a program that crashed, you did not crash the language.

Edit: or more specifically, if you put that code in a script and makes sure not to call an other functions that assumes the old behavior of your method, there should be no crash.

4 Likes
13

I agree with the points being made here. It is just a matter of optics/perception.

fatal: error thrown and no exception handler available.
MethodError(f=Int64, args=(nothing,), world=0x00000000000061d6)

Just looks very ominous. If I didn’t know any better, it looks like I just crashed the Julia internal core. At a minimum it looks like I’d have to restart the REPL.

Are fatal errors really necessary? Would it be possible to have a built-in exception handler of last resort? And if not, could we at least get rid of the word fatal. I’m just talking about perceptions here.

14

This is the exception handler of last resort. When one overrides Base functionality like this that crashes the program, you still cleanly fall back to an exception handler regardless, it’s just that the default function of this top-level exception handler is to quit. You might say that you could have it do something else instead, but what exactly should this error handler do now to recover from the actions you just took? Somehow revert the last redefinition? What if it was actually a mixture of changes that caused the program to crash? How would the error handler recover from any arbitrary error that hits it? There’s no clear solution to that, other than to quit.

1 Like
15

The fact that you see a message printed means that some code is being executed and that is the built-in exception handler and as I said, all the core functionality are working perfectly fine. There’s just little point to continue though.

2 Likes