For sure overriding methods in Base is not standard practice, sometimes packages add methods in base specializing them on their custom types.
Some of your proposals are uncharted territory, in this cases usually package developers experiment with a macro (that can have some drawbacks) and only then decide what to do (they get real feedback from users).
IMHO your proposal is fine, but it’s unusual, so make sure that you document it clearly.
Personally, I would have @nemo exp(1+exp(1)) expand into exp(MyNumber(1)+exp(MyNumber(1))), and stick to the solution @rdeits explained.
DataFrames.jl exports DataFrame so that users don’t have to write DataFrames.DataFrame all the time. You’re exporting these names to force users to write Nemo.exp. It’s technically correct as of 0.6, but it feels a bit weird.
There’s also a third solution. Keep your current exp/sin/etc. definitions distinct from Base’s, do not export them, but export instead a macro
@using_nemo_definitions()
that expands into
using Nemo: exp, sin, ...
This will shadow the Base functions. That way users who want your definitions can opt in, and won’t get ambiguity warnings just from using Nemo. Additionally, they won’t have to write Nemo.exp(1) all the time; just exp(1). They can get the Base function with Base.exp(1).
I think I should add some further explanation here, since I’ve probably not explained the reasons for wanting to do this sufficiently well.
Currently Nemo.jl is a package which provides both generic implementations of a bunch of algebraic functionality (power series, univariate and multivariate polynomials, fraction fields, residue rings, permutation groups, etc) and wrappers for various C libraries that provide highly optimised implementations over specific rings (Flint, Arb, Antic).
However, we’ve realised that if users want to apply Nemo functions to types provided by other packages, they need to build Nemo and all its C dependencies just to apply our generic Julia functionality over their types. This is inconvenient for them, and makes Nemo.jl far less useful than it ought to be.
Our idea is to separate Nemo into two components, one with all the generic functionality, written entirely in Julia, the other with all the wrappers of C libraries. We’ve actually been explicitly asked to do this by potential users. And we’ve been working towards this for some time. Most of the recent commits to Nemo.jl have been to allow this to happen in the near future.
But now we have to consider how the generic package can be tested. We can no longer write functions that test the generic Nemo power series, for example, over integer types provided by Flint! The package also isn’t useful as a standalone package, unless it can be used with types provided by Julia. This implies that we must be able to construct Nemo power series over Julia Integer and Rational types. If we don’t permit this, we make the package both useless as a standalone package, and we make it impossible to test the package properly. And our experience is that complex code that isn’t tested is broken.
Now, it is possible to just define power series over Julia Integer and Rational types separately. But now there is no way to soundly test the generic code. Of course it’s possible to construct more complicated rings to test the generic code over. But this has a far lower probability of finding bugs. This is essentially what we have been doing up until recently, and when we switched over to allowing Julia Integers and Rationals, we found about a dozen bugs in our generic polynomial module alone. This underscores the importance of testing the generic code over Julia Integer and Rational types and not writing special functions to handle this case.
Since our users won’t allow us to change the names of functions like exp, sin, inv, sqrt, etc., there really is only one possible solution that I see.
I should note that there are computer algebra systems that define exp, sin, sqrt, etc. on integer and rational types as Julia does, and there are those that don’t. But real computer algebra systems have the advantage that library code is not written in the same language as user code. That is also an option for Nemo, but then Nemo ceases to be a useful Julia module.
In symbolic systems, exp(2) returns the symbolic expression exp(2). In some systems like Python, you have to explicitly import math to get exp or sqrt defined. In most computer algebra systems 1/2 returns the exact rational value 1/2.
There’s no really easy solution here. Regardless of which solution we choose, we lose something. We can’t please everyone.
Anyway, our original question has been answered. What we want to do is not considered type piracy. The decision on what solution we choose is still pending. But so far, the best solution seems to be the one I’m proposing. And I’m pleased to find that it is permitted and won’t stop us tagging releases.
So just to clarify;
If you define module local exp and friends (e.g. don’t import them from base) you can do whatever you want with them, and they can be defined for julia types, like integers and rationals. So now users have to do Nemo.exp() and the input can be anything.
Now, if you decide to export exp users will still have to use Nemo.exp() since we need to disambiguate from Base.exp(). So exporting does not help in this case, the only thing that happens is that users of exp() must update their code to be Base.exp(). This way, by exporting you force people to change their usage of other modules, but you don’t gain anything for Nemo, since you still need to fully qualify.
Unfortunately, this doesn’t work. We need to export the exp function because users need to be able to exponentiate power series.
However, one possibility would be to have a submodule called Nemo.Math. I can’t actually see how to make this work at present. But it’s equivalent to what you are proposing without working against the module system in the language.
The problem is, I don’t see how to have the exp power series function in Nemo call Nemo.Math for Julia Integer and Rational types (when it needs to exponentiate the first term of the power series for example), but to call Nemo.exp for other Nemo types (such as other power series). It must be possible, and I will give it some thought, as this would by far be the tidiest solution!
No, this doesn’t work. The only way Nemo can export exp, even if only for power series, is to import Base. If we don’t do this, there is an ambiguity warning for any use of exp and we are back to where we were. But if we import Base.exp in Nemo, there’s no way to have Nemo use Nemo.Math.exp internally for Julia types and Nemo.exp for everything else.
So no solution along these lines can work, I think.
I’m not sure that I follow your objection, but what I proposed is this:
julia> module Nemo
export x, @using_nemo_definitions
exp(x) = "Hello"
macro using_nemo_definitions()
esc(:(using Nemo: exp))
end
end
Nemo
julia> using Nemo
julia> @using_nemo_definitions
julia> @show exp(1)
exp(1) = "Hello"
"Hello"
julia> @show Base.exp(1)
Base.exp(1) = 2.718281828459045
2.718281828459045
As I explained, we need to export exp, since this is the name of the function for exponentiating Nemo power series (a type we defined). The only way to export exp without ambiguity warnings, is to first import it from Base. Your solution doesn’t work with these constraints.
Also, there is no actual necessity to export the Nemo definition of exp(0). This is not actually useful to a user, unless they are writing functions of their own which need this definition, which is unlikely. So in that case, your @using_nemo_definitions is actually trying to solve a problem that we aren’t trying to solve.
On the other hand, there probably would be sense in exporting a sqrt function for integers that either returned an exact square root, if it exists, or raises a DomainError.
But we don’t consider it a particularly evident solution that a user should have to look up the documentation to find a funky macro they have to invoke in order to get this to work. At least just exporting our sqrt causes the user to immediately notice that there are two possible definitions of sqrt, which they can then investigate. It is much more user friendly.
Something like this perhaps?
module Nemo
export NemoType
struct NemoType end
Base.exp(::NemoType) = println("calling exp for NemoType")
exp(x::Int) = println("calling Nemo.exp for julia type")
end
resutling in
julia> x = NemoType()
Nemo.NemoType()
julia> exp(x)
calling exp for NemoType
julia> exp(1)
2.718281828459045
julia> Nemo.exp(1)
calling Nemo.exp for julia type
This would be type piracy, since we have to export exp for exponentiation of Nemo power series. The only way to do this without ambiguity warnings (which would set us right back to where we are now) is to first import Base.exp. But then defining exp(x::Int) inside Nemo would redefine Base.exp for Int, which is type piracy, precisely what we must avoid (otherwise we cannot even tag releases).
No, look at the code again, there is no export there and I can use exp for a type defined in the module withouth qualifying.
I’m sorry, I missed your previous post. I see you are just advocating not exporting exp.
I think the right answer to this is that if someone defines a Nemo power series, f say, and then does exp(f), they will just get no information. It’ll tell them exp isn’t defined. They might try exponential(f), exponentiate(f) and then give up.
At least by exporting the rogue exp, they are prompted to do something.
Edit: sorry, I see what you were saying now. Yes, this solves the problem.
Oh, right, sorry.
What? Doesn’t the code in my snippet do exactly what you want?
Oh! I totally misread your code. Yes, that absolutely does what we want!
I didn’t know you could extend a Base function without first importing it. That totally solves our problem.
Thank you very much!
@cstjean Actually, your @nemo idea could be useful to solve another problem we have. When entering a power series over Flint rationals, we used to have to do something like the following (though someone just pointed out to me we already solved this another way):
f = ZZ(1)//2 + ZZ(3)//7*x + O(x^7)
Things like this are incredibly fiddly to write. Of course Flint prints this as:
1/2 + 3/7*x + O(x^7)
which can’t then be copy’n’pasted back into the Julia REPL. (We can define custom print functions of course, but working with a system which prints all its results in such an obfuscated form is difficult to say the least.)
If we implement an @nemo macro, we could have:
@nemo 1/2 + 3/7x + O(x^7)
which would basically do what we wanted, and (mostly) solve the cut’n’paste problem, especially for very large expressions, or expressions that don’t come from Julia.
It’s still a little difficult to distinguish BigInt and Int constants when working with Julia types. But it totally solves the problem of working with Flint BigInts, which is the most common case where this occurs.
The sensible thing to do in the pure Julia part of Nemo that we plan to split off is probably to make @nemo default to BigInt’s. It’s less likely people will really want to do power series over Julia Int’s. And if they do, they don’t need the macro anyway.
Would adding a “wrapper” parametric type like this help?
struct NemoNumberType{T}
val::T
end
const NemoInt = NemoNumberType{Int}
You should then be able to define the methods in generic terms of NemoType and extending Base methods would be just fine.
We already have Nemo integer types. The issue here is about providing Nemo power series and polynomial over Julia Integer and Rational types. And I think that problem has now been solved with the solution of fredrikekre above.