With Julia 1.5 around the corner, I’m happy to present SafeREPL, a little experimental package which by default interprets Int and Int128 REPL literals as BigInt, and Float64 literals as BigFloat:
julia> using SafeREPL
julia> factorial(40)
815915283247897734345611269596115894272000000000
julia> sqrt(2.0)
1.414213562373095048801688724209698078569671875376948073176679737990732478462102
julia> c = 299_792_458; c^3 # the cube of the speed of light can even be computed safely!
26944002417373989539335912
The meaning of literals can be changed with the swapliterals! function:
julia> swapliterals!(Int => Float64)
julia> 2
2.0
So really, “safe” applies mostly only to the defaults (which is expected to be the typical use-case). But otherwise the package is “safe-agnostic”. Also, the Float64 default could be changed in the future (Float64 literals might be better off left alone).
There is also a twin package, SwapLiterals, which doesn’t touch the REPL (and therefore won’t require Julia 1.5 once registered) but provides a @swapliterals macro to swap literals in source code.
Use the following to install (Julia 1.5 is required):
using Pkg
pkg"add https://github.com/rfourquet/SafeREPL.jl:SwapLiterals"
pkg"add https://github.com/rfourquet/SafeREPL.jl"
It’s a good time for suggesting alternative package names, in particular for SwapLiterals, before these get registered.
I would like to thank the few people involved in this Zulip chat for their ideas. For example @ColinCaine suggested to use pairs instead of keywords, and @non-Jedi suggested the name literally!(Int=>Float64) instead of swapliterals!(Int=>Float64), which I find quite nice.
Feedback here or on github is very welcome.
Stay safe
It not that I don’t like this, just this existed before (was it just little know?) under: https://github.com/MasonProtter/ReplMaker.jl as you know. And maybe it’s better this way, not as a switchable REPL mode (since you can with a function), or should both be combined?
Well, indeed the possibility existed before as a REPL mode, even before ReplMaker. And honestly, I initially didn’t think at all I would publish this package, but then it became non-trivial enough to be useful to other people. Few points to try to address your questions:
The purpose of ReplMaker is to make it easy to create new REPL modes, so that package is not the right place to develop a robust “swap literals” functionality. The “Big-Mode” from ReplMaker’s README is for demo purpose, it’s not sufficient as a full replacement of the default REPL (e.g. try typing "1" in this “Big-Mode”). So I don’t think the two packages should be combined.
The core of the SafeREPL package is implementing swapping literals in the AST, not only BigInt, and not only at the REPL. There is occasionally demand for such a thing, e.g. in the zulip chat I linked to above, the OP wanted to interpret all Int literals as Int8. Some people complained that the type of unsigned literals depends on the number of digits. They can now easily have all unsigned literals be UInt64. etc.
One of the target audience of SafeREPL is beginners being confused/annoyed with how default literals can lead to wrong results. using SafeREPL is a mimimal-overhead solution, and is easier than using a custom REPL-mode: how to set-up the REPL-mode so that it’s the default when launching Julia? And also when exiting help-mode? How to enter directly help-mode when typing ?? these probably have answers, but I prefered to explore this “new approach enabled by Julia 1.5” (call it NAEBJ). It somehow seems to give more first-class status to modes that you want on all the time.
This might be more composable; e.g. I have another toy package taking advantage of NAEBJ; using REPL-modes, how can I easily use both functionalities (from the two packages) at the same time? It’s certainly possible to come up with a solution, but using NAEBJ seems easier, package authors don’t need coordination.
Anyway, NAEBJ is new and I found it interesting to exploit it; real usage will provide more elements to compare approaches.
This is some nice work @rfourquet. It’s good to see people think seriously about ways to easily provide arbitrary precision arithmetic (or other reinterpretations of literals) in a way that causes minimal friction.
Not to be defensive, but I’m a little confused about this statement:
I’m not sure what exactly you’re referring to. Could you clarify what the problem is with typing 1 in this mode? Here’s what I see:
julia> using ReplMaker
julia> function Big_parse(str)
Meta.parse(replace(str, r"[\+\-]?\d+(?:\.\d+)?(?:[ef][\+\-]?\d+)?" => x -> "big\"$x\""))
end
Big_parse (generic function with 1 method)
julia> initrepl(Big_parse,
prompt_text="BigJulia> ",
prompt_color = :red,
start_key='>',
mode_name="Big-Mode")
REPL mode Big-Mode initialized. Press > to enter and backspace to exit.
julia> >
BigJulia> 1
1
BigJulia> typeof(1)
BigInt
If I am to give you honest feedback here, I think SwapLiterals.jl is a cool and useful pacakge and if it’s used for something like SafeREPL.jl, that should be in an isolated REPL mode (not necessarily one provided by ReplMaker.jl, but I think a dependancy on ReplMaker would make your life easier ). The way you’re hot-modifying the existing, standard julia repl mode to make it evaluate code in a non-standard way makes me very uncomfortable.
To my mind, repl modes are basically a nice way to automatically apply a macro (sometimes a string macro) to code automatically. They change the meaning of code. It should always be obvious that such a thing is happening such as with a custom prompt and it should be easy to get back to standard julia when needed. This is why julia requires that macro invocations begin with the @ sigil, otherwise it would never be clear when you’re looking at code that will be evaluated normally.
I understand that you show how to add stuff to your startup.jl to make it more clear what’s going on and be able to toggle this behaviour on or off, but I think it’s a serious flaw to make those things opt-in rather than opt-out and I don’t like that they can only be done before the repl has been started up.
That’s more to do with SwapLiterals.jl though, right? Presumably something using actual repl modes could have the same relation to SwapLiterals.jl as SafeREPL.jl does, correct?
I think that having a non-standard REPL that evaluates code differently is very likely to cause other, unforeseen problems for new users that they may not be equipped to deal with if they’re unable to use repl modes. I do not look forward to trying to help someone in #helpdesk debug their code only to learn 20 minutes into the conversation that they had something like SafeREPL enabled but forgot.
Composability is an interesting problem. Thinking of repl-modes as string macros, you basically can never have the sort of composability where two separate string macros get to operate on the string, but if your transformation only needs to operate on the AST level, rather than the string level, it would actually be pretty easy to do that in replmaker. You basically just do
I could even make a method where initrepl takes in a Vector of transformations and applies them sequentially and allow people to push! or modify that vector whenever with the only caveat that the first transformation needs to take in a string, and the subsequent trasnformations need to be able to handle whatever output the previous transformation gives. Could be a String or an Expr I suppose.
So I should start by saying that I find ReplMaker absolutely great and used it a lot and will continue to do so. I actually had the intention (but forgot) to mention in my previous post that SwapLiterals could be used as a backend of a ReplMaker mode.
I hope you didn’t take my comment as a critique of ReplMaker. I was refuting the idea that “this existed before under ReplMaker”: 1) SafeREPL uses a different mechanism than REPL modes, with a different trade-off, and 2) the “BigInt-mode” from ReplMaker’s README isn’t on feature parity with what is avalaible in SwapLiterals (which can replace more literals, and is currently more robust for BigInt). The problem with "1" that I gave as an exemple was including the string quotes:
it strictly refers to the fact that SwapLiterals and ReplMaker have orthogonal purposes and must no be merged and developed in the same project. But they could/should definitely be combined by users.
I will answer that indeed SwapLiterals should be used with ReplMaker by people wanting a separate mode, but I’m wanting to explore in SafeREPL this possibility of “hot-modifying the existing, standard julia repl mode”. I totally might not foresee some problems, but applying code transformation in the normal Julia mode vs. from within a separate REPL mode doesn’t make me more uncomfortable at all.
Moreover, the ability to have REPL modes is fantastic, but I personally prefer them for non-julia code; when writing julia code, I’d rather use the julia-mode.
Well, I don’t really understand this one. Surely the package won’t force people to enable this visual indicator that code transformation is at play, but they are responsible for themselves, and the README already guides them through the steps. Also, they already have to opt-in by installing the package and using it, it’s up to them to choose the specific options.
Absolutely, I was unprecise as I didn’t want to stress there the distinction between these two new packages.
I sincerely apologize in advance if this package wastes people’s time… If this happens, a warning could be put in the README. But I will be more precise: using ReplMaker is dead-simple, so I was not implying that anyone around can not deal with REPL modes. But for the use-case we discuss here, the result is objectively not as smooth as with “hot-modifying” the standard REPL. Again, simply getting into help-mode from another mode for example requires by default to go back through the normal mode first. REPL-modes revolve around the julia> mode. It’s certainly possible to work-around that, but not for a beginner; and I couldn’t tell them how off the top of my head, even though I studied multiple times the REPL code.
So the intention with SafeREPL is to explore Julia where literals loose their absolute meaning. Yes, they have defaults, but this can be changed, in a trivial way. I typically wouldn’t use a REPL mode where integer literals are BigInt, simply because I wouldn’t bother, writing big(2) here and there is lower overhead than activating a REPL mode all the time just in case. But using SafeREPL in startup.jl offers ““safety”” by default with an overhead that I can tolerate so far (the overhead is being bothered by methods which don’t accept BigInt).
Yes, that will definitely be useful and worth implementing if there is demand for it. ReplMaker could be the “coordination” I was alluding to.
And as a last note, I’m also curious to see how this package is received by people who really really don’t like that Julia is “unsafe/fast” by default and think that this should be changed for a greater success of the language.
I am not sure how that will play out, but it will be certainly handy to be able to just refer those people to this package instead of yet another discussion of that design choice.
DecFP is supported, as shown in the README, but I’d rather not have it has default, as it’s not clear at this point that this is a better option than BigFloat (I’m not clear either on whether Float64 should be replaced by default).
So I prefer not to add this dependency (which also is not pure Julia). Also, @time using DecFP is half a second for me, 30 times slower than @time using SafeREPL: I prefer to have using SafeREPL add a very lightweight cost by default when put in startup.jl.
Good point for Decimals. However, I’m still not convinced that using decimal numbers by default rather than binary numbers is better; I guess only user-feedback would tell.
As a side note, the current printing for Decimals seems like a total no-go for the use-case of being the default in SafeREPL:
When I think about it, since your throwing speed out of the window anyway (with BigInt replacing Int), Rationals would be the best substitute for floats (just as Perl 6, now renamed Raku, uses by default, unlike Perl 5; new version also added multiple dispatch). And on display changing to Decimal or something, or some other way to get in decimal form, without like this:
julia> 1.1+1 # for your package, otherwise showing 2.1
2.099999999999999999999999999999999999999999999999999999999999999999999999999986
It just needs to be fixed in the package? I thought, maybe they want to be explicit, it’s a different type (I see pros and cons, but even DecFP doesn’t), unlike:
julia> BigInt(1)
1
FYI:
julia> Dec64(1.1) # This actually works, without your package, which has this side-effect:
ERROR: StackOverflowError:
Stacktrace:
[1] Dec64(::BigFloat) at /home/pharaldsson_sym/.julia/packages/DecFP/MVfs9/src/DecFP.jl:117
[2] convert(::Type{Dec64}, ::BigFloat) at ./number.jl:7
... (the last 2 lines are repeated 39990 more times)
[79983] Dec64(::BigFloat) at /home/pharaldsson_sym/.julia/packages/DecFP/MVfs9/src/DecFP.jl:117
Again, I don’t know that this would be best, it really depends on the use case. I think it is useful to have defaults, but it must be emphasized that the user has the choice over what is replaced by what. It’s clear that the defaults can’t satisfy everyone, so it seems premature to engage too much in discussing what are the best defaults before knowing more what users use instead of the defaults.
Personally, I typically wouldn’t want to use rationals by default, as I usually understand better the number written in decimal form; I know you suggested:
but this is out of the scope of the package, which is not in the business of modifying the output nor overriding show methods, nor defining new number types.
Yes, that’s why I said “currently”, but I would guess the authors have a good reason to have their current printing.
It’s very unlikely that SafeREPL will depend on a package for its defaults when decent types are available in Base for that purpose: it both reduce loading time, and cognitive overhead for users as they are more likely to know Base number types than these provided in a package. I could imagine using a type provided in a package if the majority of users would report using that type as their default.
I didn’t know this one, but this is indicated in the “Caveat” section of the README: many many functions assume that the user will enter a Float64 or an Int because this is the type of un-annotated literals; so it’s unavoidable to encounter such errors while using SafeREPL. I hope the situation will improve, but in the meantime you can $-quote your numbers in such situations, e.g. Dec64($1.1), or convert then back to Int/Float64, e.g. Dec64(Float64(1.1)).
No, this package does nothing to the output of the REPL, and doesn’t override package’s show methods.
In this particular case, if show(::IO, ::MIME"text/plain", x::TheType) is not defined, you might have a chance to define it outside of the package without warning. Also, without changing the default, the maintainers of the package might accept more easily a patch which change the printing depending on the value of an IOContext attribute, which you can then set in the REPL’s iocontext (since Julia 1.4).
The packages are now registered! SafeREPL depends on Julia version 1.5, but SwapLiterals requires only version 1.1.
Note that on 1.6, a nightly version of Julia is needed (unless you use the defaults, and don’t call the swapliterals! function), as there was a small bug affecting SafeREPL which was just fixed.
). The way you’re hot-modifying the existing, standard julia repl mode to make it evaluate code in a non-standard way makes me very uncomfortable.