Hi,
Rather then overloading individual operators (or functions, for that matter), is it possible to “overload” the “desuggared” function invocation? For example, supposed I have a type
type MyType
end
x = MyType()
y = MyType()
z = x + y // == desugarred_invoke(+, x, y)
function(x, y) //== desuggared_invoke(foo, x, y)
Thanks,
Tom
It’s unclear (to me) what you mean. In Julia, operators are normal functions with special syntax.
Hi Stefan,
Thank you for your reply. Let me explain in a bit more detail:
If instead of desuggaring 1 + 1 to +(1, 1), suppose it was desuggared to call(+, 1, 1), and similarly for a regular function call. foo(x, y, z) would become call(foo, x, y, z).
This, in many respects, is similar to the call overload for user-defined objects, but the issue I see is that the “first argument” is distinguished, Taking an example from the manual, for the type “Polynomial”, the overload would look more like
function call(p::Polynomial, x)
instead of the current
function (p::Polynomial)(x)
The “built-in” version would then look like
function call(f::Function, args…)
which is the actual function I would be interested in overloading.
Ultimately, I am trying to find a mechanism similar to the above where the first argument is not distinguished. For example,
abstract MyType
function call(f::Callable, x1::MyType, args…)
//Where Callable is just for illustration but typeof(+) <: Callable, typeof(foo) <: Callable and Polynomial <: Callable
or even
function call(f::Callable, args::Vararg{Union{MyType,Any}})
Does that make sense?
As an example of use consider partially applied functions:
type Placeholder
end
param = Placeholder()
pfoo = foo(1, 2, param, 3) //pfoo is now unary
I realize this may not be convincing use case but that’s actually not my problem; however, I thought it may illustrate the point.
Thanks for your time,
Tom
It’s still unclear what do you want to achieve. The call you mentioned is similar to the call on 0.4 but it is removed in 0.5 and it never applies to normal functions.
It seems that you might want a way to instrument every callsite of a function by implementing something at a different layer. Such layer doesn’t exist and doesn’t make sense to add so no it is not possible.
Hi,
Digging into this issue, I just found the following thread from back in 2014: https://github.com/JuliaLang/julia/pull/8712
It describes the functionality I’m looking for almost word for word. However, I can’t see any references to it being removed, or why. Seems like a very nice feature to have, and having the first parameter distinguished seems to break the multimethod approach Julia takes to everything else. Could you to shed some light on the background?
I am especially interested in light of your comment “that such a layer doesn’t make sense”. I have just given an example where it would be useful.
Thank you,
Tom
To me it’s still pretty unclear what you are interested in. Are you concerned with the exact syntax of function calls and type constructors? Are you concerned with the AST’s that are generated when the parser reads function calls and type constructors? Are you concerned with the semantics of the resulting :call expressions?
Even more generally, are you trying to write a function? Are you trying to write a macro?
Your reference to the concept of desugaring is, I suspect, a particular source of confusion for many because the following is how surface syntax is currently transformed into AST’s in Julia:
julia> type Foo
x
y
end
julia> Foo(1, 2)
Foo(1,2)
julia> function foo(x, y)
(x, y)
end
foo (generic function with 1 method)
julia> foo(1, 2)
(1,2)
julia> dump(:(Foo(1, 2)))
Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol Foo
2: Int64 1
3: Int64 2
typ: Any
julia> dump(:(foo(1, 2)))
Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol foo
2: Int64 1
3: Int64 2
typ: Any
julia> dump(:(+(1, 2)))
Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol +
2: Int64 1
3: Int64 2
typ: Any
julia> dump(:(1 + 2))
Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol +
2: Int64 1
3: Int64 2
typ: Any
I don’t think it does. Normal functions never go through the call. It’s only for non-function objects and the mechanism is now generalized.
It’s not removed.
The first parameter is not semantically distinct. (Apart from https://github.com/JuliaLang/julia/issues/14919). It is syntactically distinct at callsite unless we adopt lisp syntax so it makes sense to be syntactically distinct at definition too.
Your example is impossible to implement. Adding another layer will be nothing different from overloading in the current way so it shares all the problems why it can’t (and shouldn’t) be defined this way. You’ll run into issues whenever someone else try to use the same mechanism and with such a general definition, it’ll easily be overwritten by more specific methods or cause ambiguity.
Defining an closure is also something that should be clear to see from the code. It is a feature that f(a, b) won’t suddenly turn into a closure with one argument depending on the type of a or b. More technically, your Placeholder type basically can’t be passed as a normal function argument at all.
Hello yuyichao,
Thank you for your answers.
I see; that’s what I was asking about. Any references on this “generalized” mechanism?
Wait; the removal was your claim above; I just asked for more details. I am confused now.
Ok, that makes sense. I’ll have a look at the issue.
Sorry, due to the terseness of your answers I have some trouble interpreting what you’re referring to.
If you’re referring to operator overloading in general, I don’t necessarily agree. I accept your argument about the meaning of “f(x, y)”, but I feel that it’s only partially true: I can already do what I describe - say with operator +, by just regular overloading: f = _1 + 3, or something similar.
Yes, the ambiguity is a problem but I don’t think it’s any better or worse than other underspecified overloads, IMHO.
I guess the long and short of this discussion is that it’s not possible/supported. At this point, I am just trying to get a better understanding.
Thank you,
Tom
The general mechanism is
Sorry I think I misread [quote=“TomasPuverle, post:5, topic:874”]
I can’t see any references to it being removed
[/quote]
I’m referring to your
function. Which AFAICT is something doing dispatch that’s overloadable on a different layer than the current one. I’m just saying that there isn’t another such layer and adding a generic one won’t solve any problem.
This doesn’t look like valid overload/closure definition. What exactly do you mean?
These are not the same issue at all. “Underspecifying overloads” won’t cause the function to stop working.
The issue arises when you have a very loose method and still want everything to go through it no matter what other methods there are. AFAICT this is what it takes to use dispatch/overload to turn a function call into a closure definition based on the argument type and this is what won’t work.
Hello John,
Sorry if I wasn’t clear. I was trying to focus on the semantics of the desugarred ‘()’ operator.
julia> bar(x, y) = x + y
bar (generic function with 2 methods)
julia> @code_typed bar(1, 2)
CodeInfo(:(begin
return (Base.box)(Int64,(Base.add_int)(x,y))
end))=>Int64
I was asking about something like
Base.call(Base.add_int, (x,y))
In the inner expression (but even at the untyped AST level)
I think the answer below from yuyichao addresses the issue; it’s not possible. I think I will try using a macro.
Thank you for your help; it’s very much appreciated.
Kind regards,
Tom
Actually I assume you mean _1 is your placeholder?
If that’s the case then yes, you can define it now since the function overloading isn’t anything different from what you described. However, that doesn’t mean it can be defined in a reliable way or if it is a good idea to do so for reasons I mentioned above.
Also ref Function chaining · Issue #5571 · JuliaLang/julia · GitHub for some discussion essentially about changing the anonymous function syntax.
This doesn’t look like valid overload/closure definition. What exactly do you mean?
Here’s an example of what I was thinking:
julia> import Base.+
julia> type Placeholder
end
julia> _1 = Placeholder()
Placeholder()
julia> +(::Placeholder, x) = y → y + x
(This is 0.6-dev, btw)
Thanks,
Tom
Yes, I just posted an example.
I’ve been reading trough the github issues and discussions, and have, in fact found the one you just referenced. It’s important you understand that I am not proposing this is something that should be added; I am just trying to learn the language, and figuring out what’s possible. Not everything is documented (understandably), and unless you’ve been around for a while, it’s not always clear from the threads on Github if a conclusion was reached, and whether (or when) it may be implemented.
Thanks for taking the time to explain; I’ll try a different approach to my problem.
Tom
We’ve had some discussion of this sort of approach, and in Julia 0.4 there was an overloadable call function. The strange thing about such a scheme is that there’s really only a single generic function in the entire system – call. Everything else is just an argument to call, which is responsible for all dispatch. Maybe that’s good, or maybe it’s bad – it’s hard to know. Last I discussed it with him, @jeff.bezanson did not like that aspect of it. Oscar Blumberg was arguing for such an approach at some point, and I’ve been fairly sympathetic to it from time to time. What would be the concrete advantage of having a single generic call function do all dispatch?
This doesn’t seem quite like a “Usage / First Steps” kind of question. It seems to be more of a question at a language philosophy level. Unless I’m completely missing some practical aspect of it.
[Aside: If you could quote your code, that would be greatly appreciated. You can use single backticks for inline code and triple backticks for code blocks. The default style is Julia syntax.]
The idea here is to say “for f(::Placeholder), no matter what f is, do this”. Yes, that could be possible if the system used one big dispatch table where the “function” and arguments were treated equally. I think there are two major problems with this:
Practical: this definition has many ambiguities; it’s ambiguous with any definition g(x::Any).
Conceptual: this seems to violate the whole notion of what it means to define a function. When I say f(x) = blah, I believe I am defining what f means. But with the “one big call function” approach, a call f(x) might be intercepted due only to the type of x, ignoring f completely. (Really, this is just a more abstract description of problem (1) above.)
Of course, even now f(x::Any) can be superseded by a future method definition. The difference is that such a definition cannot ignore f. It’s clear “which function” you’re adding a method to. How specific the notion of “which function” needs to be is open to interpretation. I don’t know if I have a good answer, but right now our approach is that at minimum the TypeName of the function must be specified.
Do you happen to know the thread where Oscar was discussing the feature?
I was just asking about it, not arguing for/or against it. I have a lot of experience in other programming languages but not enough in Julia to feel like I can make a meaningful argument either way, as I’m not yet familiar with the “idiomatic” way of approaching problems, nor with the design philosophy behind it. The only time I tried contributing was on the “should we rename ‘type’ to something else” thread and the minute I replied, everyone else stopped, so I don’t have a good track record. 
(Having said that, I have written a number of throwaway prototype languages over the years and I really like Julia because from what I’ve seen so far, it embodies many of the ideas I aspired to with my own designs. Apart from static typing; I’ve always put dynamic types on top of a static system, not the other way around. )
One thing I was thinking was there is an element of “aspects” - the ability to execute arbitrary code around the actual function call sites - especially if there was a way to invoke the “previous” version.
As for my specific purpose, I was playing with an idea where I was trying to capture/externalize/inspect the flow of data through a function. By having the call overload, I could take a regular function and simply by providing “fake inputs” learn something about the function. Before diving into the ASTs, or trying to figure out if there is a way to access the basic blocks of the compiler, I was seeing what I could achieve at the language level just using introspection, at least for some simple cases.
I wasn’t sure where to post it; like I said, at this point I don’t know “how much I don’t know”; and before you and some of the other committers joined it (which, btw, is very much appreciated), it did seem like a “First steps” question with a simple “You can’t do that” for answer.
I will try, sorry. I am still learning the interface.
In any case, thank your for your time and input.
Tom
Hello Jeff,
Thank you for your reply,
Unless, perhaps, each function effectively generated its own unique “type”, and it would be more akin to
type g_function_unique_type <: Function #or abstract if you prefer
end
call(::Type{g_function_unique_type}, x::Any) = ...
It is sort makes me think of how lambdas work in C++; perhaps the type could actually store state, too. It seems there is already something along those lines going on anyway…
I can’t argue with that at all; it’s simply a design choice and I can’t fault it.
Thanks for your input,
Tom
It was in person now and then, usually mixed in with raucous arguments about other topics. I don’t think it ever made it to the level of concrete proposal. And it has serious problems with ambiguities and the fact that the specific function being applied can easily become irrelevant, as @jeff.bezanson pointed out.
I wasn’t sure where to post it;
Yeah, I’m still not sure where it belongs. Considering moving it to Internals & Design.
I kind-of like the view of multiple dispatch as including the function in the signature, so there is one giant method dispatch table.
Defining call on instances of abstract types would certainly be useful in a small set of cases. One could also generalize functions to allowing type parameters - I think you do see this in C++ (the new where would need to be become the only way of introducing type parameters to signatures). Not sure if that is useful, e.g. I’m not convinced of convert{Int}(1.0) rather than convert(Int, 1.0), but maybe there exists a good use-case?
Apart from those two use cases, it really seems to be mostly a conceptual difference.
Hi Stefan,
I can think of two additional things to mention besides the other comments that were already made above:
Allowing people to change the dispatch/call code may allow non-compiler developers to try and implement functional language-like features in a library setting: for example, pattern matching.
Even as an experimental playground - before proposing them to be properly integrated into the language - such a facility may be useful.
The second comment is about the function becoming “irrelevant”. Depending on your point of view, this may or may not be a valid argument. In any programming language, I can already lie:
def i_promise_i_do_foo():
return do_bar() #Haha, I lied!
In some sense, conceptually, I don’t really see the difference between
+(a, b) = explode_computer()
and
call(f, a, b) = explode_computer() #f completely ignored
Personally, I don’t feel strongly about this topic. I have, a number of times in my career, needed a similar facility, but that may have been a result of other missing features of the languages I was working with at the time.
It may be worthwhile putting this on a “wish list” to be considered later. It is likely to be a better choice to stay conservative for now. I can’t think anything that would break if in the future, the call dispatch mechanism is modified to become more general.
Kind regards,
Tom