Why does scientific notation break the range function?

[DNF]

@chadagreene

No, it’s terrible!

So, good, but not stellar?

But, replace Union{} with Integer, and I don’t really see how to improve it further. Maybe you have a suggestion?

[lmiq]

@nilshg

  A type union is an abstract type which includes all instances of any of its argument types. The empty union Union{} is the bottom type of Julia.

oh, lets be open minded here with the people just arriving to the language… There is a lot technical information in that phrase. Understanding that occurs not at the same point in Julia programming than when doing some scripts to analyze data series and plots.

(myself included, I barely understand what the “The empty union … is the bottom type of Julia” means, I remember hearing something about it from a presentation at JuliaCon about the type system and now can sort of reason about it, but I’m far to really being able to explain that)

[goerch]

@lmiq

There is a lot technical information in that phrase. Understanding that occurs not at the same point in Julia programming than when doing some scripts to analyze data series and plots.

FWIW, Racket has language levels.

[chadagreene]

@DNF

I don’t really see how to improve it further. Maybe you have a suggestion?

I’d start by identifying that the error occurs in the range function, because that’s the first thing people will want to know (yet it doesn’t appear anywhere in the current error message). Then I’d zero in on exactly where the error is within the range function, say what caused it, and conclude with some constructive advice on possible ways to fix the error.

That’s the order that people will want to process the information, and it’s the order the information should be presented in. I’d also write it in human language (a sentence) rather than a series of cryptic sentence fragments. Here’s my suggested rewrite:

ERROR in range(): Keyword argument length was entered as type Float64, but must be of type Integer.

[goerch]

@chadagreene

I’d start by identifying that the error occurs in the range function, because that’s the first thing people will want to know (yet it doesn’t appear anywhere in the current error message).

I believe this is a slight misconception: the problematic part in stack traces is in most cases the uppermost caller and not the lowermost callee.

[chadagreene]

@goerch

the problematic part in stack traces is in most cases the uppermost caller and not the lowermost callee.

What? Where in this Stacktrace does it say the error is in the range function?

Stacktrace:
  [1] top-level scope
    @ ~/Documents/coursework/Untitled-1.ipynb:1
  [2] eval
    @ ./boot.jl:373 [inlined]
  [3] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:1196
  [4] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
  [5] invokelatest
    @ ./essentials.jl:714 [inlined]
  [6] (::VSCodeServer.var"#164#165"{VSCodeServer.NotebookRunCellArguments, String})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.17/scripts/packages/VSCodeServer/src/serve_notebook.jl:19
  [7] withpath(f::VSCodeServer.var"#164#165"{VSCodeServer.NotebookRunCellArguments, String}, path::String)
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.17/scripts/packages/VSCodeServer/src/repl.jl:184
  [8] notebook_runcell_request(conn::VSCodeServer.JSONRPC.JSONRPCEndpoint{Base.PipeEndpoint, Base.PipeEndpoint}, params::VSCodeServer.NotebookRunCellArguments)
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.17/scripts/packages/VSCodeServer/src/serve_notebook.jl:13
  [9] dispatch_msg(x::VSCodeServer.JSONRPC.JSONRPCEndpoint{Base.PipeEndpoint, Base.PipeEndpoint}, dispatcher::VSCodeServer.JSONRPC.MsgDispatcher, msg::Dict{String, Any})
    @ VSCodeServer.JSONRPC ~/.vscode/extensions/julialang.language-julia-1.6.17/scripts/packages/JSONRPC/src/typed.jl:67
 [10] serve_notebook(pipename::String, outputchannel_logger::Base.CoreLogging.SimpleLogger; crashreporting_pipename::String)
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.17/scripts/packages/VSCodeServer/src/serve_notebook.jl:136
 [11] top-level scope
    @ ~/.vscode/extensions/julialang.language-julia-1.6.17/scripts/notebook/notebook.jl:32
 [12] include(mod::Module, _path::String)
    @ Base ./Base.jl:418
 [13] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:292
 [14] _start()
    @ Base ./client.jl:495

[mbauman]

That part will be improved by Why does scientific notation break the range function? - #25 by pfitzseb, but it looks like even with that change it’s missing the entry for the logrange function itself.

[goerch]

The MWE

function test(start, stop, length)
    range(start, stop, length=length)
end    

test(1, 2, 0.42)

yields

ERROR: TypeError: in keyword argument length, expected Union{Nothing, Integer}, got a value of type Float64
Stacktrace:
 [1] test(start::Int64, stop::Int64, length::Float64)
   @ Main 1.jl:2
 [2] top-level scope
   @ 1.jl:6

and yes you are right: the reference to range is missing…

But I would still search the problem either on top-level or in test…

[DNF]

@chadagreene

Where in this Stacktrace does it say the error is in the range function?

Whaaa? How on earth is range missing from the trace? I didn’t even notice that.

Anyone know why that happens?

[trahflow]

Well I guess the error is not in the range function so it makes sense that it’s not part of the stack trace.
Nevertheless, it would be nice if the error message would mention range.

A note on some relevant part of the julia documentation. It says

Keyword arguments behave quite differently from ordinary positional arguments. In particular, they do not participate in method dispatch. Methods are dispatched based only on positional arguments, with keyword arguments processed after the matching method is identified.

I think this part could be a bit more specific, i.e. it could mention how keyword arguments are processed after the matching method is identified. The way I understand it, type annotations on keyword arguments are just type assertions.

If above understanding is correct, the type assertion is probably a generic error message in base julia, which likely could easily be modified to print out the method that was identified in the preceding dispatch.

[trahflow]

Relevant issue: better error for passing wrong type of keyword argument · Issue #29829 · JuliaLang/julia · GitHub
Seems like the function/method name used to be part of the error message, but was changed. According to a comment by Jeff on the linked issue this should be part of the stack trace.
I’m confused now

Edit:
here is how the error message is formed. It has a special case for TypeErrors from keyword arguments. This is a bit weird, but I guess there was a deeper reason for that related to above issue.

[suavesito]

@jules

For example when doing divisions with divisors that are not powers of two.

If all the operations done (+, -, *, /, div) use elements inside the set of possible integers representable by a Float{16|32|64} (or a multiple of an integer with a factor of 2^{-x} with x less than the size of the mantissa) and the result of the operations is always in the end an integer, then you will get a (floating point) exact integer at the end.

But if you do something like (a / 3.0) * 6.0 with a not divisible by 3, then what you describe could happen.

[stevengj]

@suavesito

But if you do something like (a / 3.0) * 6.0 with a not divisible by 3 33 , then what you describe could happen.

Note: That wouldn’t give exact results with integer arithmetic (via (a ÷ 3) * 6) either.
Basically, any calculation that you could do purely with Integer quantities is also exact with floating-point arithmetic, up to the maximum exactly representable consecutive integer.

More generally, the key rule to remember is that elementary operations (+,-,*,/) in floating-point arithmetic behave as if they were computed exactly in infinite precision and then rounded to the closest representable floating-point value. So you only run into rounding errors when you have results (including intermediate results and inputs, of course) that are not exactly representable, like 0.1 or 2/3 or exp(3.0). This doesn’t happen with integer values up to maxintfloat.

Floating-point rounding can sometimes be difficult to analyze, but it’s important not to over-mystify it. It’s a common misconception that things like 1.0 + 1.0 will somehow give 2.0 ± ϵ in floating point.

[jules]

@stevengj

It’s a common misconception that things like 1.0 + 1.0 will somehow give 2.0 ± ϵ in floating point.

Yes that was more or less my heuristic working with floats, don’t rely on exact results. I should read a bit more about the mechanics, I’m not doing numerical work so the lowest level didn’t matter much to me. But I clearly had a few of the misconceptions you mentioned here, so thanks for clarifying.

[Volker_Weissmann]

[1] top-level scope
    @ ~/Documents/coursework/Untitled-1.ipynb:1

First line in the jupyter notebook is range(0,10, length=1e3).

[Volker_Weissmann]

@goerch

I believe this is a slight misconception: the problematic part in stack traces is in most cases the uppermost caller and not the lowermost callee.

The uppermost caller is usually something like the main() function.

[chadagreene]

That’s incorrect:

1. Notebook lines are not numbered (at least in my VS Code).

2. The first line of the notebook is NOT where the range call occurs. A Notebook cell containing this passage produces the same stacktrace message:

#this is a comment
5+5
x = range(0,10,length=1e3)

3. Sometimes a line of code may contain calls to more than one function. In such cases, it would be helpful to know that the error occurs in the call to the range() function. Therefore, I suggest that this would be a much, much more helpful error message than the current one:

ERROR in range(): Keyword argument length was entered as type Float64, but must be of type Integer.

[Volker_Weissmann]

@chadagreene

The first line of the notebook is NOT where the range call occurs. A Notebook cell containing this passage produces the same stacktrace message:

Ok, then that’s a real issue.

[StefanKarpinski]

This is a good point! Suppose float lengths were allowed. Then if you wrote some code that computes the length of a range as a float and just happens to end up as an integer in your test cases. But then when you use your code in anger it doesn’t happen to compute an integer length anymore. Bam! It errors because the float cannot be converted to an integer value. In the current version on the other hand, you are forced to convert to explicitly convert to integer somehow even if the value happens to be integral, alerting you to the fact that this can fail or that rounding or flooring or whatever is necessary.

Of course traditional dynamic languages consider this situation above fine. Stuck a string in an array of integers? Guess you’ll find out eventually! But Julia tries to catch these things earlier by encouraging correct use of types. It catches it later than static languages but much earlier than untyped dynamic ones.

[liuyxpp]

I happened to find that Union is quite intuitive: when I first came across it, I get the idea and understand it without googling the doc.