Is there any reason why Julia needs me to fully qualify the function name when the argument types are clear and there shouldn’t be any problem with multiple dispatch?
Case 1: load JuliaDB before defining Foo
julia> using JuliaDB
julia> stack(
stack(t::D) where D<:Union{IndexedTables.NDSparse, IndexedTables.NextTable} in IndexedTables at /Users/tomkwong/.julia/v0.6/IndexedTables/src/reshape.jl:32
stack(t::D, by; select, variable, value) where D<:Union{IndexedTables.NDSparse, IndexedTables.NextTable} in IndexedTables at /Users/tomkwong/.julia/v0.6/IndexedTables/src/reshape.jl:32
stack(t::Union{JuliaDB.DNDSparse, JuliaDB.DNextTable}) in JuliaDB at /Users/tomkwong/.julia/v0.6/JuliaDB/src/reshape.jl:7
stack(t::Union{JuliaDB.DNDSparse, JuliaDB.DNextTable}, by; select, variable, value) in JuliaDB at /Users/tomkwong/.julia/v0.6/JuliaDB/src/reshape.jl:7
julia> struct Foo end
julia> stack(f::Foo) = 1
ERROR: error in method definition: function IndexedTables.stack must be explicitly imported to be extended
Case 2: define Foo before loading JuliaDB
julia> struct Foo end
julia> stack(f::Foo) = 1
stack (generic function with 1 method)
julia> using JuliaDB
WARNING: using JuliaDB.stack in module Main conflicts with an existing identifier.
Thanks for the references but I don’t fully understand the design decision here. The method signatures are clearly different for the two functions and they shouldn’t be any conflict.
Consider this example:
julia> module A
export foo
struct ABC end
foo(x::ABC) = 1
end
A
julia> module B
export foo
struct XYZ end
foo(x::XYZ) = 2
end
B
julia> using A
julia> using B
julia> foo(A.ABC())
WARNING: both B and A export "foo"; uses of it in module Main must be qualified
ERROR: UndefVarError: foo not defined
Further, if these functions are not coming from different modules then multiple dispatch works beautifully. Why is importing (or using) any different?
julia> bar(x::A.ABC) = 1
bar (generic function with 1 method)
julia> bar(x::B.XYZ) = 2
bar (generic function with 2 methods)
julia> bar(A.ABC())
1
julia> bar(B.XYZ())
2
Multiple dispatch only works for different methods of the same function. Your example consists of two completely different functions that happen to have the same name. To add methods to an existing function in another module, you always have to import that function or use its fully-qualified name when defining the method. Otherwise you’re just creating a completely new function which also happens to be called foo.
This is intentional because the alternative would be chaos. The requirement to use an import or a fully-qualified name clearly indicates that you want to extend an existing function with a new method. If that requirement were eliminated, then completely unrelated packages would end up conflicting with one another because both Module A and Module B defined foo(::Int). The result would be type piracy everywhere.
Thanks for the detailed explanation. I think this statement covers it well as far as the mechanics of multiple dispatch.
Perhaps it should be a feature request. IMHO, defining a function with my own types (hence not a type piracy problem) that happens to have the same name as an exported function from a dependent package should still be allowed. There’s no ambiguity.
It is? But obviously, if you use two packages that export two different things with the same name, there will be a conflict. If you wanted to extend the function, then do so.
I agree that it wouldn’t be a type piracy issue if Julia let you define your own stack function in this way (which would be separate from the stack function exported by the dependent package), so then locally the identifier stack would point to your function rather than the existing one.
The trade-off is that if you actually did intend to extend the existing stack, things are broken in a subtle and confusing way. I don’t have any data on this, but my hunch is that extending functions is far more common than defining new functions with the same name as an existing one. I’m guessing this error is thrown to prevent that situation.
Note this is also an argument for using Foo: bar, baz to be explicit about what names you pull into your local namespace, so if you did want to define your own stack function, you wouldn’t using it.
Actually I find none of the answers convincing. I met this problem in my code and I think there is no good solution. I have a package implementing polynomials and there is naturally a degree method for them.
Then later I wrote a package for permutations and I wrote a method degree for them (for a permutation degree is how many points they move). I can use one or the other package without problem. But, If I use
both together I got the same dreaded message about the need to qualify the function.
The only workaround I found is to have a dummy package which defines the function and have each package extend the definition from dummy. I find this a ugly hack.
I think the only reason that the current situation in julia is iivable is because quite often the function one
wants to extend (like the arithmetic operations +, *, / etc…) is in base. Then it is not painful that each package imports from base then extends.
A solution would be if one could add names to base. Otherwise I do not know what is a good solution.
It’s pretty common for ecosystems of related packages in Julia to have an “xBase” (like StatsBase.jl or DiffEqBase.jl that defines shared functions and types for different packages in the ecosystem to extend. These packages also can be points of collaboration between people working in related domains.
I agree it’s a trade-off and takes some coordination between the different packages that use the shared functions. The alternative of automatically merging together functions with the same name has been discussed before, but that has trade-offs of its own.
I’d definitely be curious to hear if you have a proposal for a better solution.
I think I suggested a possibility: the possibility to add names to base (just adding empty definitions).
At least then one would not have to invent a dummy name for a glue package.
Think about the implications of this suggestion. The space of function names is (practically) infinite, so a large and asymptotically growing part of Base would be
function a_function_that_two_people_use end
export a_function_that_two_people_use
function another_function_that_is_kindof_handy end
export another_function_that_is_kindof_handy
The exports are optional (your proposal did not specify — and of course we would use macros ).
The current solution of explicitly importing (and resolving ambiguities when necessary) is much better. There are always trade-offs between verbosity, ambiguity, and clashes, but the status quo is kind of a sweet spot:
it works without a hitch if there is no overlap between exported symbols,
asks for explicit action when there is one,
and distinguishes using and extending functions by requiring the latter to be explicit.
Also, some packages choose not to exportany of their interface (eg ForwardDiff.jl) and require explicit imports, which is also a valid design choice.
I find this a very nice and clean solution. Packages are very lightweight in Julia, making it very easy to make and use small ones. Many collections of packages do this, and it will become even nicer with Pkg3.
Perhaps computationally lightweight, but maintaining small packages is a chore. They feel like C++ header files: redundant work.
Haven’t followed Pkg3. Does it have a solution to this problem?
IMHO the best solution would be to allow defining Polynomials.degree(m::MyType) = ... without having to fully import and REQUIRE Polynomials. Then we wouldn’t need interface-defining packages. But it’s not trivial design-wise.
My understanding is that it will make it easier to specify (potentially unregistered) dependencies, and has faster download/install times. This should make these small extra packages essentially unnoticeable by the user, and very cheap for the developer.
I don’t understand how it is redundant in this instance. This is something that needs to be specified by the developer, the language cannot infer this otherwise.
If I understand correctly, Pkg3 would allow multiple packages with the same name and/or version to coexist. So I don’t see how a detailed specification of the requirements could be avoided.
Perhaps we’re talking about two different things. I’m talking about a package like ScikitLearnBase.jl, whose main content is a bunch of forward declaration.
function fit! end
function predict end
function is_classifier end
...
Those functions already have full definitions in ScikitLearn.jl. Why do I have to define them in ScikitLearnBase.jl again? We only go through this because package X wants to be able to support package Y without imposing Y’s full loading time on X users who don’t care about Y. If packages were as fast to load and as lightweight as Python packages (… a tall order, I know), X would just import Y.
Author A develops package A (say “polynomials”) exporting function degree and then author B develops
package B (say “permutations”) exporting function degree. If user C wants to use both packages he has
no way to use degree without qualification. If authors A and B are the same person she can think forward and
make a package dummy_declaring_degree and have both A and B importing and extending degree from this dummy package. If authors A and B are different persons and do not communicate C is screwed up.
I think is best to abstain from hyperbole: the worst case scenario is the user having to resolve the ambiguity manually (via using/import, declaring abbreviations for package names with const, etc).
If two packages share verbs (functions) that are similar, the best case scenario is that the maintainers cooperate to put these in a third package. This does happen in practice, eg see
Eventually, all these issues boil down to people being reasonable and cooperating when applicable. I don’t think this is a bad outcome, and more importantly, it is not clear how one can improve on this while preserving separate namespaces.
This discussion reminds me of a best practice when writing Java code. Import statements should be written explicitly so the namespace isn’t polluted with unnecessary dependencies. Here’s a sample reference post.
If I would explicitly import specific functions from the dependent package then there’s no conflict. Ideally, a good IDE would auto generate the using statements for me. When I use Eclipse for Java, it’s a single click.
julia> module A
export foo
struct ABC end
foo(x::ABC) = 1
bar() = 2
end
A
julia> module B
export foo
struct XYZ end
foo(x::XYZ) = 3
end
B
julia> using A: bar
julia> using B
julia> foo(B.XYZ())
3
julia> bar()
2
).