Allowing the object.method(args...) syntax as an alias for method(object, args ...)

[bertschi]

Nice example of how an argument can be inserted at any position. Just to try out the new syntax, I have hacked together a small macro which rewrites ++ (a valid and as of now unused operator) into right-associative calls of minusminus (our higher-order function corresponding to the invalid operator --):

using MacroTools: postwalk, @capture

minusminus(obj, meth) = (args...; kwargs...) -> meth(obj, args...; kwargs...)

function unchain(terms::AbstractVector, stack::AbstractVector)
    if isempty(terms)
        :(foldr(minusminus, [$(stack...)]))
    elseif @capture(terms[1], f_(args__)) && f != :foldr  # don't touch nested transforms again
        arg = :(foldr(minusminus, [$(stack...), $f])($(args...)))
        unchain(terms[2:end], push!([], arg))
    else
        unchain(terms[2:end], push!(stack, terms[1]))
    end
end

macro calumet(expr)
    postwalk(expr) do ex
        if @capture(ex, ++(args__))
            unchain(args, [])
        else
            ex
        end
    end
end

Both of your examples now work as follows:

julia> @calumet "Hello, world!"++split(",")++uppercase++map()++join(":")
"HELLO: WORLD!"

julia> @calumet let x = [1,2,missing,4,5,missing]; x++skipmissing()++isodd++filter() end
2-element Vector{Int64}:
 1
 5

Also nested transformations should work, but I have not tested extensively:

julia> @calumet 1 ++ 2 ++ Base.:+() ++ (3 ++ 4 ++ Base.:+()) ++ Base.:*()
21

[adienes]

Are /> and \> available where one could curry into the first argument and the other the last, otherwise using the same semantics as -- ? or would this be too confusing

[uniment]

@bertschi

Just to try out the new syntax, I have hacked together a small macro which rewrites ++ (a valid and as of now unused operator) into right-associative calls of minusminus (our higher-order function corresponding to the invalid operator --):

Holy wow that was fast! You’re a wizard! Admittedly I began looking into writing a macro to do this, but you beat me to the punch by a wide country mile

This is really cool! Although, two complaints:

• I’ve begun to use the ++ operator for the parallel sum, as seen here.
• This macro doesn’t yet work for broadcasting a function across a collection, e.g. [1,2,3]++sqrt.().

I know I know, eventually we’d settle on dash --, I’m pulling your leg.

This is really great. I like the “peace pipe” reference too.

Masterful. I’m going to play with this for a while, and maybe some other folks can play with it too and see what else it’s good for (and what it’s not).

@adienes

Are /> and \> available where one could curry into the first argument and the other the last, otherwise using the same semantics as -- ? or would this be too confusing

Similar to dash --, /> and \> are not valid operators, which means we would need them to be added into the language first; it also means we could claim them without breaking backwards compatibility in any packages or programs, if we can convince the Julia devs to incorporate them.

There was a brief moment when I was considering emdash --- for piping the object into the last argument position, such as arr---map(sqrt). If it has otherwise the same properties as we have chosen for dash -- (higher precedence than function call, and right-associativity), then the following is true:

f(a,  b,  c,  d) ==
d---f(a,  b,  c) ==
c---d---f(a,  b) ==
b---c---d---f(a) ==
a---b---c---d---f()

This operator would be nice for map, reduce, filter, etc., and especially well-suited for mapreduce, e.g.:

[1, 2, 3]---filter(iseven)---map(sqrt) ==     # using emdash
    [1, 2, 3]--iseven--filter()--sqrt--map()  # using dash

[1, 2, 3]---mapreduce(x->x^2, +) ==        # using emdash
    [1, 2, 3]--(+)--(x->x^2)--mapreduce()  # using dash

And if dash -- and emdash --- have the same precedence, then they should chain as expected, e.g.:

[1, 2, 3]---mapreduce(x->x^2, +)--sqrt() ==
    [1, 2, 3]--(x->x^2)--map()--(+)--reduce()--sqrt() ==
    sqrt(mapreduce(x->x^2, +, [1, 2, 3]))

However, I’ve begun thinking that dash -- is sufficiently expressive to cover the vast majority of use cases, that the ease of typing it beats the other options we’ve considered (except perhaps ..), and that adding more and other operators to serve these purposes would bring more confusion and mental loading than it’s worth.

But I’m open to the idea that I’m wrong. Maybe you can convince me?

[uniment]

Playing around a bit, this could get really confusing

As specified previously, let dash -- and emdash --- be defined as:

--(obj, meth) = (args...; kwargs...) -> meth(obj, args...; kwargs...)
---(obj, meth) = (args...; kwargs...) -> meth(args..., obj; kwargs...)

meaning that -- partials an object into the first argument position and --- partials an object into the last argument position. Make them bind more tightly than function call, be right-associative, and have equal precedence such that:

f(a, b, c, d)       ==

a--f(b, c, d)       ==
b--a--f(c, d)       ==
c--b--a--f(d)       ==
d--c--b--a--f()     ==

d---f(a,  b,  c)    ==
c---d---f(a,  b)    ==
b---c---d---f(a)    ==
a---b---c---d---f()

It gets really weird if you start intermixing them; you end up with a lot of different ways to call a function:

f(a, b, c, d)       == 

d---b--a--f(c)      ==
c---d---b--a--f()   ==

d---a--f(b, c)      ==
b--d---a--f(c)      ==
c---b--d---a--f()   ==

a--c---d---f(b)     ==
b--a--c---d---f()   ==

a--d---f(b, c)      ==
c---a--d---f(b)     ==
b--c---a--d---f()   

And although they are numerous, they are not arbitrary; arguments can only be picked off the argument list from the left end or the right end, one at a time. (Note: for a single-argument function, arg--f() and arg---f() are equivalent.)

This seems so insidious that I’m leaning toward disallowing it, and banishing the emdash --- operator altogether in favor of having only the dash -- operator. As mentioned previously, most well-designed functions have whatever object you might wish to chain in the first or second argument position anyway; even if it might be somewhat uglier to use -- for calling map or filter, the Pandora’s box that --- opens up isn’t worth the convenience imo.

Heck, I’d rather use the half dozen piping packages than live in a world where -- and --- coexist.

Please try to convince me otherwise!

[uniment]

I’m coming around to the idea of having both operators, and I’m starting to like @adienes’ idea to use /> and \>. Apologies for changing my mind so quickly.

New Proposal (Compare with old proposal)

Two infix operators, /> (“fill” or “frontfill”) and \> (“backfill”), defined as

/>(obj, meth) = (args...; kwargs...) -> meth(obj, args...; kwargs...)
\>(obj, meth) = (args...; kwargs...) -> meth(args..., obj; kwargs...)

which bind more tightly than function calls, have right-associativity, and have equal precedence to each other.

These operators generate partially applied functions, calling the method to the right on the object to the left—frontfill /> filling the object as a first argument and backfill \> filling the object as a last (non-keyword) argument—operating as follows:

my_obj/>meth1(args1...)/>meth2(args2...)/>meth3(args3...) ==
    meth3(meth2(meth1(my_obj, args1...), args2...), args3...)
my_obj\>meth1(args1...)\>meth2(args2...)\>meth3(args3...) ==
    meth3(args3..., meth2(args2..., meth1(args1..., my_obj)))

and can be used in conjunction with each other for chaining operations:

"Hello, world!"/>replace("o"=>"e")/>split(",")/>uppercase.()/>join(":") ==
    join(uppercase.(split(replace("Hello, world!", "o"=>"e"), ",")), ":") ==
    "HELLE: WERLD!"

"1, 2, 3, 4"\>eachmatch(r"(\d+)")\>map(x->x[1]\>parse(Int))\>filter(iseven)/>sum() ==
    sum(filter(iseven, map(x->parse(Int, x[1]), eachmatch(r"(\d+)", "1, 2, 3, 4")))) ==
    10

Why am I changing my mind? Because as coders we already have plenty of ways to make code illegible, yet we don’t because it doesn’t help us. We might as well offer both tools—frontfill and backfill—and trust that we won’t abuse them.

Although /> and \> are slightly more awkward to type than --, it’s not too bad. Especially if you type them a lot because it’s actually a useful feature, you get good at it (also anybody who’s typed a lot of XML should already be good at typing />). In addition, they also exhibit stylistic cohesion with the existing pipe operator |>. (may or may not be a good thing.)

I’m really excited to watch autocomplete work with this and pop up dialogs with matching method signatures.

[NiclasMattsson]

I confess to having been somewhat exasperated by this thread. First it was apparent shoehorning of unmotivated OOP syntax into Julia, then it seemed as if Julia-is-not-at-that-stage was being willfully ignored, and finally the -- and --- operators were proposed. Maybe it’s just me, but I find those operators almost comically terrible, as if developers of the Brainfuck language had snuck into our forum and were drunk-posting just to troll us. And at first I felt the same way about /> and \>.

But on further reflection they’re actually starting to look pretty interesting and useful. As you say, /> and \> work together with |> as an extended family of piping operators. Yes, they can be abused to make illegible code but so can everything. And I don’t think newbies will find them more confusing than existing argument transformation syntax like f() do … end blocks.

So I’m tentatively going to click like on your proposal because I hope people like me who’ve been averse to this thread will notice and comment on it - especially those with insight into Julia’s parser who can clarify whether this is at all feasible. And kudos to everyone involved for persisting!

(But consider editing your post again to add spaces around the pipes. Legibility is everything, and you need this to be attractive to sell it.)

[Palli]

@NiclasMattsson

But consider editing your post again to add spaces around the pipes.

IF we add such operators, should spaces be mandatory (I use them and feel better with piping in Unix, but it would be too late, or breaking to require for older |>)? There’s one precedent for space required for operators, a ? b : c, and while space/indentation is not meaningful in general in Julia, if I recall there’s one more such corner case. [Plus style-guides recommend spaces in some cases, and against in others.]

[DNF]

@Palli

if I recall there’s one more such corner case.

Maybe

julia> 2 .+(1:3);

julia> 2.+(1:3);
ERROR: syntax: invalid syntax "2.^"; add space(s) to clarify

?

[haberdashPI]

I share similar feelings to @NiclasMattsson

I think this is starting to turn into a more and more interesting idea for a higher-order operator over functions. So, first of all, thanks for your civil discourse and thoughtfulness.

However the ordering of arguments in [1,2,3]--iseven--filter()--sqrt--map() is pretty confusing as a reader. I think the properties of this operator are just too unintuitive in the current proposal.

I think this is much clearer: [1,2,3] -- filter(iseven, _) -- map(sqrt, _), and there is already a proposal for tight binding of _ to create anonymous functions (https://github.com/JuliaLang/julia/pull/24990); though it remains unclear if that will ever land.

[adienes]

I think it is slightly more consistent with |> if

A /> foo(args) is simply an alias for A |> (x -> foo(A, args)) and
A \> foo(args) is simply an alias for A |> (x -> foo(args, A))

Not sure if I’m totally sold on all the other semantics of --. It seems like it is trying to do too many things.

[bertschi]

Well in a sense, A /> foo(args...) acts like A |> x -> foo(x, args...), but is also a higher-order function in its own right, i.e.,

A /> foo(args) == (A /> foo)(args)  # by precedence
               == ((xs...) -> foo(A, xs...))(args...)  # by definition of />
               == foo(A, args...)  # by beta reduction

Together, the two operators /> and \> are quite powerful and basically allow to pass arguments one-by-one to a function, either from the front or back respectively. Thus

f(a, b, c, d, e) = (a, b, c, d, e)

⬚ -> f(1, 2, ⬚, 4, 5) == 2 /> 1 /> 4 \> 5 \> f

and in a chain we can use a suitable sequence of /> and \>to insert the starting argument at any desired position (together with regular function application either one of the operators is sufficient already; when functions cannot be called regularly, both would be needed). Taking this further, tacit or point-free programming would also be possible, i.e., use the operators to fill all but one argument and then compose the resulting functions.

When the syntax hinted at by @haberdashPI is available there is no need for these operators, as we can simply use the regular |>, i.e.,

[1,2,3] |> filter(iseven, _) |> map(sqrt, _) == [1,2,3] |> ⬚ -> filter(iseven, ⬚) |> ⬚ -> map(sqrt, ⬚)
                                             == [1,2,3] \> filter(iseven) \> map(sqrt)
                                             == [1,2,3] /> iseven /> filter() /> sqrt /> map()

Overall, those operators might be more interesting theoretically than practically. As a first step, one could maybe implement a VSCode plugin that hides ⬚-> such that ⬚ -> f(x, ⬚, z) just shows up as f(x, ⬚, z) … just like the Lisp programmer I once meet, who didn’t like brackets and had an Emacs mode not showing them – Scheme looks surprisingly much like Python without brackets .

[uniment]

@NiclasMattsson

Maybe it’s just me, but I find those operators almost comically terrible, as if developers of the Brainfuck language had snuck into our forum and were drunk-posting just to troll us.

Thanks for the compliment! I can only wish to be so lucid

@Palli

IF we add such operators, should spaces be mandatory

Spaces are only mandatory in settings where behavior would otherwise be ambiguous; I don’t see how that’s the case here, though I could be missing something.

I’m currently hanging onto hope that good syntax coloring would smooth out the appearance. Maybe a mistake?

@haberdashPI

However the ordering of arguments in [1,2,3]--iseven--filter()--sqrt--map() is pretty confusing as a reader. I think the properties of this operator are just too unintuitive in the current proposal.

The current proposal is no longer proposing --, but instead /> and \>.

What we find though, emerging simply as a surprising result of the operators’ currying properties, is that [1,2,3]\>filter(iseven)\>map(sqrt) is interchangeable with [1,2,3]/>iseven/>filter()/>sqrt/>map().

I’m still mulling over the consequences of this. The obvious benefit is that if a function or functor takes many arguments, but you want to chain into just the second one, there’s a mechanism for it: arg2/>arg1/>func(args[3:end]...).

@haberdashPI

I think this is much clearer: [1,2,3] -- filter(iseven, _) -- map(sqrt, _), and there is already a proposal for tight binding of _ to create anonymous functions (https://github.com/JuliaLang/julia/pull/24990 ); though it remains unclear if that will ever land.

In your example, you’d want to use the standard pipe operator:

[1,2,3] |> filter(iseven, _) |> map(sqrt, _)

But yes, _ curry underscore could make piping into arbitrary argument positions work nicely. In agreement with stevengj, I suspect the only implementation that could gain traction is tight evaluation (one call only). If I’m not mistaken, tight evaluation would mean:

[1,2,3] |> y->filter(x->x≠2, y) |> y->map(x->x^2,y)  ==
[1,2,3] \> filter(x->x≠2) \> map(x->x^2)             ==
[1,2,3] \> filter(_≠2) \> map(_^2)                   ==
[1,2,3] |> filter(_≠2, _) |> map(_^2, _)

thus rendering /> and \> mostly redundant with |> and _.

@adienes

I think it is slightly more consistent with |> if

A /> foo(args) is simply an alias for A |> (x -> foo(A, args)) and
A \> foo(args) is simply an alias for A |> (x -> foo(args, A))

Not sure if I’m totally sold on all the other semantics of --. It seems like it is trying to do too many things.

It’s not trying to do so many things, just those capabilities emerge naturally as a result of its properties.

If we define it as you propose instead, many of those capabilities go away. Is it a good thing? I’m not sure.

@bertschi

Overall, those operators might be more interesting theoretically than practically.

Perhaps.

Afaict, the benefits that /> and \> could bring over |> with _ are:

• to capture the object into a partially applied function, e.g. f = my_obj /> meth, leaving the other arguments unfilled
• to allow one or more arguments and their locations to be chosen before beginning to type the method name, which should be beneficial for autocomplete to match the method signature in realtime.

I know some folks downplay the importance of autocomplete, but it does reduce mental loading quite a bit.

Oh yeah, forgot the most important benefit of />:

• most natural feel for OOP programmers, e.g. my_obj /> meth(args...)

Okay okay that was a joke, don’t hurt me. I know we don’t care about the OOP folks.

But yeah, tight _ currying does attenuate the impetus for /> and \>. I’m curious to see where the cards land.

[uniment]

Seeing that /> and \> as proposed are mostly redundant with |> with _ as argued, the instinct is to choose one.

For the following discussion I will refer to tight _ currying as “placeholder syntax”, and /> and \> as the “fill operators.”

Considering that placeholder syntax and fill operators work to make different things convenient, the question becomes which things ought to be more convenient. Here’s an overview of the differences.

Piping/Chaining

Placeholder syntax requires a couple more characters, but can curry into any argument location as easily as any other:

x |> filter(iseven, _) |> join(_, ", ") |> myfunc(a, _, c)

Meanwhile the fill operators require a tad less thought, until you want to curry into a position that’s not at the front or back of the argument list.

x \> filter(iseven) /> join(", ") /> a /> myfunc(c)

So the question is raised: How frequently should we wish to chain an object into the first or last argument, versus arbitrary locations?

Extra Comments:

It can be argued that a benefit of the fill operators will be to encourage smart argument orderings which place important arguments near the front or back of the argument list.

Also, it seems like the fill operators will likely make autocomplete work better, as method signatures can be searched with one or more arguments already specified while the function name is typed out.

If you’ve tried ?(x, <tab> in the latest Julia REPL you’ll know it’s not a good experience (but maybe that’s temporary).

Partial Function Generation

Both placeholder syntax and fill operators yield a partially evaluated function.

Placeholder syntax requires all-but-one arguments to be known and yields a single-argument function:

f(_, a, b, c) ≈ x->f(x, a, b, c)

while the fill operators do the opposite, requiring one argument to be known and yielding an all-but-one argument function:

x /> f ≈ (a, b, c)->f(x, a, b, c)

These roles seem complementary in a way.

Similarity with Other Languages

Similarity with other languages shouldn’t be the primary reason for a feature, but it can be instructive to look at what has already proven useful.

The proposed placeholder syntax functionality is similar to Scala’s placeholder syntax.

Meanwhile, the fill operators’ functionality is a superset of Clojure’s relevant threading macro behavior (also Lazy.jl’s threading macros); it is also a superset of D and Nim’s UFCS behavior, as well as the relevant behaviors from OOP languages’ dot-notation method calling.

Scala has both: placeholder syntax and OOP-style dot-notation method calls.

Thoughts?

P.S. To play and experiment with these options, we currently have:

• Placeholder-like syntax: DataPipes.jl, Pipe.jl, Chain.jl, and Underscores.jl (there are many caveats, nuances, and differences, depending on the package)
• Clojure-like threading macros: Lazy.jl
• Fill operators: @bertschi’s experimental peace pipe macro (frontfill only, using ++ instead of />, no broadcasting for now)

Let me know if I missed anything!

[bertschi]

Just a minor point, _ would be partial application and not currying. The difference is subtle though and Wikipedia somewhat confusing on that issue. In a nutshell, currying refers to the transformation of a multi-argument function into higher-order functions with a single argument each, i.e.,

f(x, y, z) = x+y+z         # multi-arg form
h = x -> y -> z -> x+y+z   # curried form

Curried functions are particularly suited for partial application as arguments can be passed one-by-one, i.e., z -> f(1, 2, z) vs h(1)(2) as the corresponding forms for partial application.

Ok, enough wisenheimer. As most of the discussion seems to be about a concise syntax for partial application now, i.e., either using _ or the fill operators, I start liking this notation:

[1,2,3] |> ⬚->filter(iseven, ⬚) |> ⬚->map(sqrt, ⬚) |> ⬚->sum(⬚)
# or if you must use a macro
macro λ(app) :(⬚->$app) end
[1,2,3] |> @λ filter(iseven, ⬚) |> @λ map(sqrt, ⬚) |> @λ sum(⬚)

After all there must be a reason why Julia supports Unicode .

[uniment]

@bertschi

After all there must be a reason why Julia supports Unicode .

Unfortunately, I cannot support this proposal. I don’t even know which character is ⬚.

Okay I found out it’s \dottedsquare. Maybe if it had a shorter alias…

[uniment]

Looking at Base.Fix1 and Base.Fix2, there’s a lot of overlap between the proposed operators and the internal fix functions. If Fix1 were opened up to accepting more than one argument, it’d be completely equivalent to />. And for two-argument functions, \> is equivalent to Fix2. In other words, the proposed operators /> and \> are simply the logical extension of Fix1 and Fix2.

The only thing missing would be parametric types. Perhaps introduce parametric types FixFirst and FixLast, and then make /> and \> syntax sugar for them?

For example, in namedtuples.jl line 104 I see:

NamedTuple{names, T}(map(Fix1(getfield, nt), names))

with the proposed operators this could be written as

NamedTuple{names, T}(map(nt/>getfield, names))

and presumably, you’d see something like this:

julia> nt = (a=1, b=1.0);

julia> typeof(nt/>getfield)
Base.FixFirst{typeof(getfield), NamedTuple{(:a, :b), Tuple{Int64, Float64}}}

julia> nt/>getfield(:b)
1.0

similar for \> of course:

julia> arr = [1, 2, 3];

julia> typeof(arr\>map)
Base.FixLast{typeof(map), Vector{Int64}}

julia> arr\>map(x->x^2)
3-element Vector{Int64}:
 1
 4
 9

As mentioned, having a parametric type that embodies partial application

@Tamas_Papp

exposes the components of what would be an otherwise opaque closure, and allows neat things like a show method for certain function/argument combinations

One can also conceive a Base.Fix{n} where {n isa Int} type… or for the truly deranged, Base.Fix{k} where {k isa Symbol} for currying into keyword arguments. I’m not sure what’s a good way to make a type that would cover underscore currying.

[xgdgsc]

I’ m working on an implementation like something in Goland in vscode extension. If anyone want to try out, please see Release A quick dot methods completion demo · xgdgsc/julia-vscode · GitHub . Welcome opinion/bug reports.

[uje]

We are thrilled to announce the release of REPLference.jl, which was created in response to this thread. After months of development and testing, we are proud to present a package that solves the problem initially presented by @gianmariomanca in this thread. This will make it easier for beginners to learn and approach Julia (and hopefully it solves your problem @gianmariomanca).

It is worth noting that this package was not developed overnight. Instead, we undertook a rigorous and comprehensive process of testing every object with every name generated by the script that generates all documented names in the Julia programming language. This ensured that we created a package that is comprehensive and returns every name that can reasonably be called on an object.

However, we acknowledge that this is just the beginning and that we need the support of the wider community of experts and contributors to refine and optimize this package to meet the needs of beginners who are just starting with Julia. We welcome feedback and ideas from those interested in contributing to the continued improvement of this package.

Our ultimate goal is to provide an easy learning experience for beginners entering Julia, and with the help of the wider community, we are confident that we can achieve this. So, we invite you to download and try out this package and let us know how we can make it even better.

[Henrique_Becker]

This is very good indeed, it basically brings the (spirit of the) old unix man to inside the Julia terminal working with all the already existing machinery.

[uniment]

@uje

hopefully it solves your problem @gianmariomanca

It doesn’t seem to work for the OP:

julia> using REPLference

julia> f = IIRfilter(0.9);

julia> man(f)
ERROR: MethodError: no method matching man(::IIRfilter)

julia> fun(f)
ERROR: MethodError: no method matching fun(::IIRfilter)