Fixing the Piping/Chaining Issue (Rev 3)

[uniment]

Quite creative, I like it!

@Jollywatt

The main selling point seems to be repurposing brace syntax { } to be function composition with built-in support for partial application.

There is a lot more to this proposal, but I’m not so sure about the rest of it.

…

I also prefer Chain.jl ’s use of _ as the anonymous argument over it or any alphanumeric name.

I think there’s a certain nuance that’s missing though, which is:

@uniment

I think the ultimate objective should be to find settle on a syntax which can be a proper part of the language, so that the effort to build out the other things that would depend on it can be justified.

Namely, I’m trying to create a concept which is as general as possible, and as a result, trying to avoid the behavior of threading the argument into a default position. Additionally, as has been expressed previously, the use of _ is somewhat wasteful, when within the local context it is possible to define new keywords—this would keep _ free for use in partial application, as PR #24990 proposes.

Indeed, if this proposal were to be accepted and #24990 were accepted (which is my hope), then many of the expressions here could look closer to Chain.jl:

"chains".{
    split(_, "")
    _.^2
    join(_, "•")
    uppercase
}

@Jollywatt

Personally, I find x |> {f, g} better than x.{f, g}

The other points that this misses are: 1) creation of an unneeded lambda (increases compile time) and 2) operator precedence. It’s frequent for chains to be short sub-expressions inside larger ones (e.g., arr.{filter(f, _), first}.a+1), and the lower precedence of |> forces more of it to be placed within the chain, e.g. arr |> {filter(f, _), first, it.a+1}, which is clumsier. The high precedence of . is useful.

@Sukera

typing { on my german keyboard requires pressing ALTGR (on the right side of the spacebar) as well as 7 (0 for }) in the number row (also on the right side of the keyboard).

Oh dear, now I understand the dislike for curly braces! How prevalent are such keyboards?

[Barget]

@uniment

Oh dear, now I understand the dislike for curly braces! How prevalent are such keyboards?

As far as I know, belgian and french too…

[uniment]

Another point I think bears re-emphasis:

A large motivation for pushing for acceptance as a language feature, is to get autocomplete support. Not only would this improve method discoverability as the OO folks have, but it would also make it so that threading into a default argument position (first for Chain.jl, last for DataPipes.jl) isn’t as compelling: for example, if this proposal and PR #24990 were accepted, then in "1:2:3".{split(_, ":")}, an autocomplete would likely fill out (_, ) and place the cursor after the comma, saving the effort of typing the underscore.

(Before I catch flak for the compile time of constructing a partial applicator, in proposal #2 I proposed giving _ preferred treatment within the chaining syntax so that it would be syntax-transformed into a simple function call. I didn’t restate that in this proposal, but I do still carry that intent. Perhaps I should code it in.)

As a result, I feel inclined to oppose the behavior of automatically threading into any default argument position, because the effort it saves would be negligible.

@Barget

As far as I know, belgian and french too…

This is not good. How do they manage C-style languages? And in Julia, are they much less inclined to use type parameterization?

I suppose if we went forward with this proposal, an autocomplete would become popular very quickly: when typing obj., probably the first option to appear should be obj.{} so that these characters need not be typed. (of course, they’d be only entered if you hit <tab>.)

[antoine-levitt]

@uniment

This is not good. How do they manage C-style languages?

we suffer.

[Barget]

@uniment

This is not good. How do they manage C-style languages? And in Julia, are they much less inclined to use type parameterization?

Well, it’s not that bad. For belgian, thumb and index at aligned (see link). But for french, indeed, that’s more painful …
Belgian keyboard layout
French keyboard layout
Bottom line, keep in mind the vast majority of languages use curly braces, and this doesn’t prevent these countries (Germany, France, Belgium, other …) to code in e.g. javascript / C / Go / etc. …

[sijo]

I did too, until I decided to do all coding with a US keyboard layout. It works well because in code files I write everything in English anyway. And every OS has a standard keyboard shortcut to switch layout for writing documents in French/German/etc. I’m much happier ever since, also having less strain in my fingers when coding.

[Palli]

[Off-topic? I.e. this post, as some others, only about typing on different keyboards, e.g. braces.]

That applies to e.g. the Icelandic keyboard too! For me at least it feel very natural to type in { and }. I suppose if people opposed very much Java, C and C++ and other curly-brace languages wouldn’t be popular (in Germany and some places)…

You do get people occasionally asking for this in Julia instead of begin … end. I’m very pro on not doing that (and it will not happen), not because of your objection, but it’s more useful for other things. Would this new syntax be most useful way of explaining those brackets?

@Sukera

That’s more difficult to type than @ or |>

Well @ for me is AltGr and Q so not easier to type with one hand… (I never understood why the other Alt no [allowed to] working, since it does nothing) as I type the braces. I suppose with two hands as I do slightly easier than the braces with one (or two hands, if I were to adjust to doing that).

@Barget

it’s not that bad. For Belgian

Wow, that’s an awful (looking, e.g. for M) keyboard, with ( typed as, I suppose AltGR and, 5 (plus it being AZERTY, at least odd to me); or maybe not I guess you no longer for for using one hand (which was my first thought when responding, since I’m used to that).

[adienes]

I agree with this summary. I would be curious to see the reception of the simplest parts of this, which is more or less “Chain.jl but with braces,” and with the differences you propose since I also prefer each of those, submitted as an actual PR into Base. It seems there is sometimes a different nature of discussion that happens on GitHub than on Discourse.

[Barget]

@Jollywatt

julia> "chains" |> {
           split("")
           _ .^ 2
           join("•")
           uppercase
       }

I like this approach very much … exepct the REPL customization part, which makes it non-easily reproducible / deployable. But it’s understandable due to its sheer experimental nature.

Building on top of that, I was considering that maybe a more Julian-consistent syntax could be

julia> "chains" pipe
           split("")
           _ .^ 2
           join("•")
           uppercase
       end

… where pipe (or similar) should be a reserved keyword, such as let, begin, do, etc.

[Sukera]

@Barget

Bottom line, keep in mind the vast majority of languages use curly braces, and this doesn’t prevent these countries (Germany, France, Belgium, other …) to code in e.g. javascript / C / Go / etc. …

And it also doesn’t prevent us from developing carpal tunnel due to that We can’t change existing languages, but I’d rather not repeat the mistakes of the past when other solutions already exist.

A bit offtopic, but if you’ve ever wondered why Vim uses the ESC key for so many operations even though it’s very far out of the way - on the keyboard it was developed on, it was very accessible:

Also note the easy-to-press curly braces and pipe symbol (both left-shift+KEY (right-shift works too, but left-shift makes it once again easier with both hands)), which just aren’t an as prevalent thing on other keyboards…

[uniment]

If anything, the discussion over the last few comments has made me happier about the constraint to have the . dot in x.{f, g}, as it offers information for an autocomplete to fill in the curly braces {}.

@Barget

I was considering that maybe a more Julian-consistent syntax could be

"chains" pipe
    ...
end

… where pipe (or similar) should be a reserved keyword

That seems like an interesting idea. I would avoid it for short chains, e.g. arr.{first}.b[end], but for multi-line expressions it could be nice to have a more verbose syntax. This would be consistent with how the expression (a; b) is equivalent to the block expression begin a; b end.

I don’t know enough about parser design to know how difficult this would be to add to the language. But I like the idea.

[Jollywatt]

@uniment

I would avoid [block syntax] for short chains, e.g. arr.{first}.b[end], but for multi-line expressions it could be nice to have a more verbose syntax.

In my opinion, not being worth it for small chains is an advantage. It would encourage people to just write

b(first(arr))[end]

plainly. Suddenly having a short, alternative spelling like a.{length} for every function call length(a) would inevitably lead to unfortunate bifurcation of style across the Julia codebase, if it were to become syntax.

Don’t let me shoot you down, though. I realise x.{f} is more amenable to autocompletion, etc. (And this objection doesn’t apply if it were just inside a macro.)

[uniment]

@Jollywatt

Suddenly having a short, alternative spelling like a.{length} for every function call length(a) would inevitably lead to unfortunate bifurcation of style across the Julia codebase, if it were to become syntax.

I suspect this might be one of the bigger hurdles: overcoming the notion that it’s simply “not Julian” to place the function name in suffix position instead of prefix position, or that accommodating two styles would be too confusing.

Maybe I’m just an optimist, but I don’t think it’s bad. I see it as similar to how we are free to say, “he measured the length of the baby” length(a) or equivalently “he measured the baby’s length” a.{length}. The latter might invoke an image of a man with a baby, pulling out a measuring tape; while the former might invoke an image of a man with a measuring tape, stumbling across a baby to measure. They could both be true at various instants, but the one we go with is whichever makes more sense in the context.

(and using the pipe operator a |> length is like saying, “he took the baby and fed it into a length measuring machine.” )

@Jollywatt

And this objection doesn’t apply if it were just inside a macro.

Indeed, many things about this proposal would be easier if I simply settled on releasing a macro. My fear is that, if all I do is release a macro, then there is insufficient motivation for someone to develop an autocomplete that leverages it.

[jules]

Regarding non-Julian things, I considered at the time to have @chain x { ... } but decided against it because it seemed bad to introduce a competitor block syntax to begin end in a macro I could see used widely across the ecosystem, just to save a few characters. Which was the same reason I called it @chain and not something more cryptic like @> or other options. I thought that would make it easier to search for as well if you didn’t know the macro.

[jar1]

Personally I think { ... } would be ideal to replace all the ... end blocks in Julia. begin end is way too long for common block macros like chain, but IMO its replacement could be general to all macros, rather than privileging the chain use case.

[aplavin]

let-end is a bit shorter than begin (:
DataPipes supports both (begin and let) for pipes, with corresponding differences in variable scopes.
The let-end semantics is arguably much more common and natural in this context, begin-end is only for cases when multiple variables are defined in the pipe and used later outside of it.

[adienes]

so proposal 4 I am seeing scattered over a few comments: do all of the following

1. merge #24990 to get underscore partial application
2. merge Chain.jl into Base with three changes
   a. make @chain no longer thread automatically into first argument
   b. change @chain x begin ... end semantics to simply x chain ... end or x pipe end
   c. (maybe?) allow links in the chain to be separated by commas or pipes than only newline

[Barget]

FWIW, getting just 1 & 2 even without change would be already great (I find the syntax proposed by Chain.jl it already useful)

Also,with 2.b) I realise the way to capture the output is not obvious. So at least, this aspect should be carefully thought through.

[adienes]

@Barget

I realise the way to capture the output is not obvious

maybe x |> chain ... end is better

[uniment]

@jar1: We already have a shorter alternative to begin a; b end for blocks: (a; b)

@jules: Appreciated. Considering that {...} parses differently than blocks anyway, that was a good move imo.

I do like the idea of having a more-verbose block syntax for multi-line {...}. Is there any analogy here? I don’t believe I’ve encountered any more-verbose block syntax for [...].

The let-end semantics is arguably much more common and natural in this context

@aplavin: I agree, I don’t like side-effects here; that’s why I’ve chosen to make x.{f, g} mean let it=x; it=f(it); it=g(it); it end.

This also makes its scope behavior consistent when you say: chain={f, g}; x.{chain}, because {f, g} means it->(it=f(it); it=g(it); it)

I’ve actually even gone a step further, and made it so that any local assignments have the local keyword prepended—that’s how much I dislike side effects

@adienes and @Barget: The thing you must understand, is that the behavior of underscores in PR#24990 is incompatible with their behavior in Chain.jl. For example, PR#24990 would propose that map(_^2, _) should create a function that behaves like x->map(y->y^2, x), whereas Chain.jl treats it as if it’s x->map(x^2, x).

You have two possible paths:

1. Adopt underscore behavior like Chain.jl, which then makes it impossible to use _ for partial application (at least, if we are to demand any consistency in language behavior).

2. Adopt underscore behavior like PR#24990. This allows underscore partial application both inside- and outside-of chains.

You choose path (2), as it offers the ability to write _+1 instead of Base.Fix2(+, 1), as well as opening up all sorts of other partial application applications (e.g. if a function composition fallback is implemented, you could write filter(_%3==0, arr)).

However, you soon run into the problem that you can’t make expressions where the same variable appears more than once: _+_^2 doesn’t behave as x->x+x^2 and instead throws an error. But you say, “This is a chain! Of course I meant the result of the last expression!”

You might go back and decide that you prefer (1). But that would prevent partial application syntax outside of chains, and you would burn _ on something which didn’t need it (remember that we’re creating a new context, in which we are at liberty to choose new keywords!).

So then you say, okay. Let’s choose path (2), and select a local keyword. You survey the landscape of identifiers. You consider ⬚, but it’s difficult to type. Then you realize, singular non-gendered object pronouns carry the exact meaning that you are after here. So then you arrive at it, and the exact behaviors of this proposal.

Which brings us back to:

@uniment

Can you explain your reasoning for continuing to oppose it, so I can better understand your viewpoint?