Function name conflict: ADL / function merging?

Internals & Design
,
101

OK, I will.

But I hope you also consider that for 5 years, a group of people who probably know a median number of 8-10 languages each has been iterating on the design of the language and sophisticated packages. But while there are aspirations for Julia to take on matlab and python, so far I would guess that you have very few users who only know matlab/python and then go Julia.

During that time, if ChrisR finds an incompatibility with a package that Tamas is working on, they would just scan through the source-code, find the offending problem, fork it on github, and chat on slack… This is not scalable, and requires a huge degree of sophistication from people using libraries to scan for the function, inject into another namespace, etc. Chris and Tamas are not mortals (and yes, I am picking on your two out of respect) where people like me are! If I copy a bunch of example code with using and suddenly I get incompatibilities, then I am confused.

I think Julia has already shown that it is many people’s favorite 5th, 6th, 7th, 8th, etc. language. But I don’t think it has shown that it can be people’s first language, and things like this make a big difference. Teaching an introductory course with Julia was eye-opening. What is good for package developers (e.g. fully qualified functions, etc.) is not necessarily what is good for users.

1 Like
102

No, this is the single-dispatch example. No issues at all, because which f() you mean is unambiguous:

class MyType1:
    def f(self):
        return 1

class MyType2:
    def f(self):
        return 2
    
    
mt1 = MyType1()
mt1.f()

mt2 = MyType2()
mt2.f()

Similarly single-dispatch in C++ works like this. And operator overloading in C++ does the same thing for functions… unless it is ambiguous from the types, at which point it errors. The issues with C++ overload resolution are primarily that the generic programming was patched onto it later, but you don’t have the same issue in a multiple-dispatch language.

1 Like
103

I would be happy if the qualification of the names would be required whenever inference
cannot determine which f to call. This would cover more than 99% of the use cases
I can think about. And more in the future when inference improves.

1 Like
104 
struct MyType1
    f::Function
end

struct MyType2
    f::Function
end

m1 = MyType1(x -> x +1)
m1.f(1)
m2 = MyType1(x -> x +2)
m2.f(1)

Using objects as namespaces can already be done with e.g. getfield overloading in 0.7.

105

If I liked single-dispatch, I would use python. My point was the lack of ambiguity, not that I like calling everything as a member function.

106

I don’t think you appreciate quite how annoying and unpredictable this behavior would be. Worse still, it would be completely unreliable across Julia versions—the notion that type inference only becomes more precise over time is wrong. Since compile time is a major issue right now, one of the ways the compiler is likely to improve in the near future is by becoming better at deciding when not to bother with inference and specialization. (For example, #26834, opened just today.) There are many situations where a better compiler is one that knows less about types. This suggestion would prevent us from improving the compiler without breaking people’s programs in a way that this thread’s very existence proves that they would find quite annoying.

6 Likes
107

Automatic merging is a great source of accidental type piracy. You define f(x::MyNumber) and then f(x) to catch everything else. Someone else defines f(x::Number) for god knows what reason. Now your functions which use f on Numbers are changed without any warning as to why, it just acts differently when package B is around! Of course the answer is that package A should merge in package B’s f if they don’t have some kind of overlapping dispatches. When package C usings them both do you get a warning to use namespaces? It sounds like it ends up at namespacing really fast if you think about it like that.

3 Likes
108

Sure, that can happen, and I agree that no amount of ambiguity detection could fix it (because there is no ambiguity!)… but if you tell the major package developers to be disciplined in not treading on types outside of their library, then it will be a relatively rare occurrence. I suspect very little confusion from beginners on that one - in part because they are not likely to write much of their own generic code.

But if the alternative is we need to tell new users to choose one of: (1) use import instead of using and then qualify everything with namespaces; (2) use using but when you have trouble using incompatible libraries, or want to use a function name already used, then teach them how to recognize the problem and how to manually merge the types; or (3) deal with this type of type piracy in the rare moments they use it from undisciplined libraries… Then I pick (3)

Anyways, I am not going to convince anyone else at this point. @StefanKarpinski , thank you for discussing the technical difficulty of why it is tough to implement in a dynamic language, which I appreciate. Tough to implement is different from all sorts of “lisp curse”-style responses on how Julia is so flexible I could do it myself.

4 Likes
109

I don’t doubt that there are things we can do to be a better first language. But people get used to the little flaws in whatever languages they know well, and rationalize them away. For example, I once heard a professor complain that python can be confusing to teach because some functions are top-level (f(x)) and some are methods (x.f()). I suspect most python programmers never think of that as a problem. The name conflict issue discussed in this thread is, perhaps, one of these things on our side. But I don’t think julia is somehow uniquely difficult to learn out of all high-level languages.

I’d be interested in other feedback from your experience teaching julia, if you have more items.

4 Likes
110

Please do start a new thread for other feedback though since I wouldn’t want it to get lost at the end of this already somewhat long thread :laughing:

2 Likes
111

Thank you everyone for your opinions and inputs!

@StefanKarpinski, thanks for being patient with us all along.

@Jean_Michel and @jlperla, thanks for articulating this request so well.

I strongly believe that such discussions are quite healthy and positive, and as usual I learn a lot from conversations like this.

This community is one of the reasons why I like to continue working with this language – either professionally or recreationally.

10 Likes
112

Thanks. Although teaching people to call member functions on objects and to call top level functions otherwise is a whole different ballgame than helping them understand why they are unable to create a function with the name they want, or what to do if they can’t use 2 packages at the same time that otherwise work fine individually.

I will post up other issues in a separate thread in a few weeks, but most of them have to do with limiting the complexity of example code, and the well-known environment issues people are already working on (e.g. plots, plots, Pkg speed, plots, the lack of the debugger/code stepper, and plots).

113

Namespace conflicts and their solution (use namespace-qualified names) are pretty universal in programming languages, so I don’t see why Julia would pose a unique pedagogical challenge here.

1 Like
114

@StefanKarpinski Thank you for your detailed reply. It is much appreciated.
.

expressions don’t have types, values do

That’s a nice explanation of Julia’s dynamic types. Thank you, I think I understand that now:
Methods are dispatched on run-time values, and if type-inference works this can be static, otherwise Julia uses dynamic dispatch. Is this correct?

I see a parallel between auto-qualification and multiple-dispatch. It seems the current situation is run-time method dispatch, but compile-time function dispatch. Is this true?

It seems to me that the issues of “auto-qualification” are exactly the same issues of method dispatch. Would it be possible for Julia to have “auto-qualification” based on run-time values? If types can be inferred at compile-time then qualification happens at compile time and all is good. Otherwise we have dynamic qualification at run-time. Of course the latter is not ideal and is a performance “gotcha”. But this performance gotcha applies to ordinary method dispatch too.
.

Agreed. That 5% is a problem. But I also think the 5% of the time that ordinary method dispatch is not statically inferred also causes some confusion and pain. Many posts on this forum are related to performance issues from code that cannot be statically type-inferred.
.

Exactly.
Alternatively, just as you can annotate code with type-assertions, you could still explicitly qualify functions.
.

Rather than a new function f, couldn’t A.f or B.f be determined dynamically? Isn’t this case an example of dynamic dispatch anyway?

Similarly for map. The seemingly magical type-inference of map remains a mystery to me. However, couldn’t the same techniques be applied to auto-qualification?
.

One final note. My questions are purely academic, to gain a better understanding of the issues at play here. I am not necessarily advocating for “auto-qualification”. I don’t have sufficient knowledge or experience for that.

Again thank you for your time and patience!

1 Like
115

I need to thank every body for the explanations, too. But the issue is not resolved. I hope we get back to it.

I heard from knowledgeable people in this thread that “meaning” is fundamental to the language decision on this issue. Is there any other area of the language where the decision is also based on “meaning”?

116

Yes. All of it. The choice of variable names and when to put a _. The choice of function names. The choice of when to put together variables into a struct. Higher level languages are written to encode human-legible meaning into computer programs because that’s how it becomes legible, succinct, and interpretable. Every choice of interface, object or type structure, recursion setup, etc. is done because its meaning makes it easy to interpret. A programming language where all sense of meaning is stripped out is simply an assembly language.

6 Likes
117

These are not part of the design of the language, but of the system build by users (which are the same persons as the designers of the language when building the core library) using the language. What could be still part of the design of the language is guidelines how to name things if they are enforced. For instance
it seems that the guidelines on when to use _ or concatenate English words without _ are not set in stone,
so this would be more be a matter of taste than part of the language.

At the core, since the language is to be interpreted by a computer which needs well-defined concepts to execute a program, only the concepts which can be given a mathematical definition are really part of the language as I see it.

118

Don’t try to use mathematical purity as some escape (ever.). Math isn’t related and its not pure. Julia has structs. You can put anything in there, but if they aren’t matching meaning then it’s not a useful program structure. Julia has arrays, but if you aren’t putting like things in there then the fact that they are in a collection is not useful program structure (you can put all of your variables into one Array{Any}!). Julia has multiple dispatch, but randomly composing everything into one function is not a useful program structure. Julia gives you a structure to work with, but if you take out all meaning then your code is not interpretable. The underlying “mathematical definition” doesn’t matter.

2 Likes
119

Well, there is somewhere a definition which tells the computer how to interpret the program. This is what I call “the language”. This definition can be formalized, thus is of a mathematical nature.

There is no such definition which tells how to use the language. Of course all the things you mention are
needed to use intelligently the language. But a feature of good languages (of which I think Julia is one) is
that clever programmers can make a use of them which was not even imagined by the designers.

1 Like
120

“Meaning” here exists in a more formal sense, as well as in a fuzzy human-level sense. For example, when a variable has a certain value, the variable can be said to “mean” that value, in the sense that the language semantics say to “interpret” or “evaluate” a variable by looking up and returning its assigned value.

The next relevant step here is that a generic function is an object that holds a collection of methods, and that can be called. It is in fact a mutable object; you can add methods to it after it’s constructed. When you write function f end (and there’s no existing f) two steps actually happen: a generic function object is created, and the variable f is bound to it. That is why two variables with the same name can mean different things: because they can point to different function objects.

Because generic functions are mutable, and because function equality is undecidable in general, functions are compared “intentionally”, by identity. So if you have two function objects, to julia they are just different, regardless of whether they implement the same human-level concept or even whether they have identical methods. So if we care whether two functions are equated, it’s on us to make them one function object in order to express that in the language.

The issues described in this thread arise because there’s no automated way to make the underlying formal meaning correspond to the intended informal meaning — we have to specify manually in each case. That can be tedious, and the question is whether some mechanism exists that could make it easier.

11 Likes