Function name conflict: ADL / function merging?

Can you be more specific and show me? I am here to learn…

This discussion reminds me of a best practice when writing Java code. Import statements should be written explicitly so the namespace isn’t polluted with unnecessary dependencies. Here’s a sample reference post.

If I would explicitly import specific functions from the dependent package then there’s no conflict. Ideally, a good IDE would auto generate the using statements for me. When I use Eclipse for Java, it’s a single click.

julia> module A 
             export foo
             struct ABC end
             foo(x::ABC) = 1
             bar() = 2
         end
A

julia> module B
              export foo
              struct XYZ end
              foo(x::XYZ) = 3
          end
B

julia> using A: bar

julia> using B

julia> foo(B.XYZ())
3

julia> bar()
2

Consider

module A
export foo
foo() = :A
end

module B
export foo
foo() = :B
end

Possible solutions would be

  1. import one, using the other

    import A
    using B
    A.foo()
    foo()
    
  2. import both, quality with module name

    import A
    import B
    A.foo()
    B.foo()
    
  3. if the module names are too long, abbreviate

    import A
    Renamed = A  # in practice, this would be an abbreviation
    using B
    Renamed.foo()
    foo()
    

You can combine 2 and 3.

I see no solution in your post. I want a way to do:

julia> using Polynomials
julia> using Perms
**** some magic that you tell me how to write ****
julia> a=Polynomial(1,2,1)
x^2+x+1
julia> b=Perm(1,4,3,2)
(2,4,3)
julia> degree(a)
2
julia> degree(b)
4

without needing to write Polynomials.degree and Perms.degree in the last lines.
And assume I am not the author of the packages Polynomials and Perms so I cannot modify them.

3 Likes

FWIW regarding this particular example: Polynomials.degree and Perm.degree which does not do the same thing at all, and therefore should probably be separate functions, and not extend som common function. Seeing degree(x) in code would be much more confusing rather than Polynomials.degree(x) and Perm.degree(x).

That two packages export the same name does not make it unusable, you just have to prefix with the module. IMO this is also much clearer for persons reading the code.

10 Likes

Another option for this particular case:

module A
export foo, AA
struct AA end
foo(::AA) = :A
end

module B
export foo, BB
struct BB end
foo(::BB) = :B
end

module C
import A, B
import A: AA
import B: BB
foo(x::AA) = A.foo(x)
foo(x::BB) = B.foo(x)

@show foo(AA())
@show foo(BB())
end

i.e., just define your own function. You could probably automate this process using a macro and some introspection. But I agree with others that having functions from different packages be automatically ‘merged’ would probably lead to some very confusing behavior.

4 Likes

There are some mathematical contexts where you can see degree doing the same thing.
I could parody your explanation:

*(a::Int, b::Int) and *(a::Matrix, b::Matrix) which do not do the same thing at all (commutative and noncommutative multiplication). You should use the qualified names Base.* and LinearAlgebra.*.
Seeing A*B in code would be much more confusing rather than LinearAlgebra.*

I hope you get my meaning… Anyway the argument “does not do the same thing” is not pertinent. Let’s take another example: the first package is UnivariatePolynomials, the second one is MultivariatePolynomials.
Then degree clearly does the same thing. Next you will come up with an argument that one should not use the two packages together.

I would prefer answers which tackle the problem rather than sidestep it.

8 Likes

OK, your code is a solution but shows how painful such a solution is.

I don’t ask that “functions from different packages be automatically merged”, but that there exists a way or
a convention in writing packages which would allow to merge them.

1 Like

I’ve been playing around a bit, and it might be possible to even set up a module in Main (via eval called in __init__) if it doesn’t already exist), and add empty function definitions there (if not already present), and then import that function from that module, and then extend it. If multiple packages used the same approach, they could dynamically share function names, without worrying which package gets loaded first (as long as the functions only operate on type from their own module)
I’m not sure yet whether that will work, but it might be a solution to your problem.

3 Likes

I agree with @Jean_Michel. Usually, when this issue is discussed, a common response is that there is no problem at all, perhaps a minor inconvenience. “You should qualify names anyway” is a common rejoinder. Or “coordinate with other packages” (!) Its clear to me that this is a problem. Some downsides of multiple dispatch tend to get explained away. (This problem does not occur with objects that own their methods.)

“You should qualify…”. Yes, sometimes I want to qualify names, sometimes I don’t, and its a PITA if I have to. If I use a function from a package once or twice in a large source file, it can really improve clarity to qualify it.

But, this is different

 julia> using DataFrames
using 
julia> using DataStructures
WARNING: using DataStructures.tail in module Main conflicts with an existing identifier.
WARNING: using DataStructures.head in module Main conflicts with an existing identifier.

If I want to use head and tail at the REPL with both these packages, what am I supposed to do ? This example involves just two, widely used, packages. So, yes, a solution can be cobbled together. But, what about all the other future (or current packages) that want to use head. Are they really different functions or are they methods of the same function ? Maybe there will be varying shades of semantic similarity for the functions. If my package’s use of head is semantically close enough, I’m supposed to rely on coordination with all the others. If it is not close enough, I am supposed to chose a different name.

Of course it is neccesary for functions of the same name in different Modules to be different functions, rather than methods. Namespaces are necessary. But, it doesn’t follow that the functions are semantically different. Or that need for namespaces plus multiple dispatch does not sometimes create problems that are not well localized. The issue will become more important as the number of packages grows.

3 Likes

I’ve been playing around a bit,

This is interesting. Would it require coordination between packages ? I assume this would only be used in cases of a possible collision. In the future, when there are thousands of packages, this could be difficult logistically. Maybe the mechanism could be dynamic, so that a package only imports a name from another package if it is already loaded. But, there will be ambiguities:

f(x, y::A)   # in PackageA
f(x::B, y)   # in PackageB

Agree :100:%

Would it be even better if such module is in Base? Taking it to the extreme, let’s say we take every verb/noun in English dictionary and create empty functions in Functions_EN.jl. Then, ask package developers nicely to extend from it. If possible, Julia should just auto-extend from it i.e. if I’m exporting length then Julia should be smart enough to make my length functions extend from Functions_EN.length.

There was a very long discussion of this issue: #2025

1 Like

I have to admit that I still don’t understand the issue. I don’t think this is an issue with function overloading in C++. For example

namespace A{
    struct AA{};
    auto f(AA){
        return 0;
    }
}
namespace B{
    struct BB{};
    auto f(BB){
        return 1;
    }
}
int main()
{
    using namespace A;
    using namespace B;
    f(AA());
    f(BB());
}

Is multiple-dispatch inherently incapable of implementing “overloading” in this sort of way? (Yes, I know that C++ is a mess, and that overloading isn’t done with dispatching …)

The fundamental problem is that semantic difference and similarity is not something that the language can decide; it is up to the programmer(s).

Manual resolution of namespace clashes (either by the user, or the authors of the relevant packages) is the only viable solution I have seen so far.

I agree that the issue will become more pronounced as we have more packages. Not exporting anything (eg ForwardDiff), or exporting a subset (Optim), is also an option.

2 Likes

My C++ is a bit rusty, but if you try to compile

#include <iostream>
namespace A {
  int f(int x) {
    return x;
  }
}
namespace B {
  int f(int x) {
    return x + 1;
  }
}
int main()
{
  using namespace A;
  using namespace B;
  cout << f(1);
}

you get the ambiguity warning. I think that C++ tries to be clever about selecting the namespace based on the method signature, and just fails when it can’t do that.

Would this be a reasonable solution for Julia? I am not so sure. C++ is compiled, so it can see all the calls before deciding what to do. In contrast, Julia can be used interactively, so it cannot decide whether there will be a conflict just by two namespaces sharing a function at the time of importing them. Just dealing with it then looks like the most reasonable solution to me, but maybe there are other options.

Actually I would like to discuss this assumption which is at the origin of the currently discussed problem. I find that in general julia is very powerful because it does not impose arbitrary restrictions on what the user can do. This is an exception. Is this exception really necessary? I remind you that in some languages like Ruby users can override the built-in
semantics, redefine + on integers to mean something else, and still the language does not
fall apart and this redefinition/extension of the standard library is quite powerful.

So, what catastrophy would happen if “functions from different packages would be automatically merged”?

2 Likes

True, but that really is ambiguous since you are trying to call f on the same type (int). The example I gave (where the function is overloaded on different types ) is not? Moreover, the underlying types are defined in completely separate namespaces.

Unless I am missing something, isn’t this the issue with the function name conflict? That you can’t have include the same function name for 2 totally unrelated types created by different packages?

Consider e.g.

module A
export foo, bar
foo(x) = println("everything in order")
setup_code_a(x) = foo(x)
end

module B
export foo
foo(x::Int) = Base.trigger_world_war_3()
end

If a user is used to working with A:

using A
setup_code_a(1) # everything in order

but has decided that B maybe has some functionality they want and so tries:

using A, B
setup_code_a(1) # death is upon us

This is overly dramatic of course, but I think this example displays a key characteristic of ‘automatically merging functions’, namely that it can introduce bugs that have non-local causes. The user called setup_code_a, but the root cause of the problem lies in foo, which the user may be completely unfamiliar with. Worse, the bug may only occur for certain argument types. I think that ‘failing fast’ in the presence of functions with the same name from different modules (the current approach) is much easier to debug.

8 Likes

You can do this.

Method overwrites for example.

3 Likes