Fixing the Piping/Chaining Issue (Rev 3)

You’re not allowed to make tuples with missing elements separated by commas, either.

As for why not to use tuples, I offered this:

The way it works, you can use tuples in single-chains if you wish to avoid multi-chains, but I don’t see how it makes sense to impose that constraint on yourself when multi-chain syntax is more succinct and [potentially] parallelizable.

To me the broadcast syntax is an issue, presumably this would be used on collections a lot, and having to change the order of things or the syntax to broadcast looks quite arcane to me ({it+1, abs2}.((0,1,2,3)), (0,1,2,3).{{it+1, abs2}.(it)}).

This seems too nice to pass on (even though it might not be possible to implement currently):

f(x)    # single element   
f.(x)   # broadcast      

x{f}    # single element     
x.{f}   # broadcast

Maybe just a small bug :

julia> @mc {it.x = 2 - 1, sin(it.x)}(r)

0.8414709848078965

julia> @mc {it.x = 2 -1, sin(it.x)}(r)

**ERROR:** syntax: unexpected comma in array expression

I agree! Unfortunately, x{f} is already claimed (for a pretty important feature of the language :wink:), or I definitely would have taken this approach. I’m open to ideas.

Wow I wasn’t expecting someone to run into this so fast. :sweat_smile: It’s an artifact of how arrays are processed, which is slightly different than normal syntax outside of arrays:

julia> [1 - 2] # 1-element column vector
1-element Vector{Int64}:
 -1

julia> [1 -2] # 2-element row vector
1×2 Matrix{Int64}:
 1  -2

julia> [1-2,] # 1-element column vector
1-element Vector{Int64}:
 -1

julia> [1 -2,] # error
ERROR: syntax: unexpected comma in array expression

It throws an error here because , commas aren’t supposed to appear in matrix definitions.

We can also see it in another context where you might not expect it: space-delimited macro arguments.

julia> @show(1 -2) # comma-delimited b/c inside parentheses
1 - 2 = -1
-1

julia> @show 1 - 2 # one argument
1 - 2 = -1
-1

julia> @show 1 -2 # two arguments
1 = 1
-2 = -2
-2

Another mildly unexpected behavior:

julia> abstract = 1; type = 2;

julia> [type abstract]
1×2 Matrix{Int64}:
 2  1

julia> [abstract type]
ERROR: syntax: unexpected "]"

I wasn’t expecting anyone to bump into it so fast. It’s simply how the parsing machinery works for [] matrices, which extends to {} (but *not* () tuples or blocks). I don’t think it’s terrible, considering how easy it is to avoid (i.e., don’t name variables abstract and type, and avoid writing - in a way that suggests the use of the unary operator unless it’s intended to make multi-chains), but it’s possible to have unexpected behavior, and it can be pretty non-obvious why an error is being thrown.

Maybe worth a note? Or? I’m open to ideas.

1 Like

This is a fun thread! But I want to draw attention to the heart of the proposal.

What we want

  1. Easy-to-type, left-to-right chainable function composition

    Like x |> f |> g or x.f().g() instead of g(f(x)).

  2. Easy-to-type partial function application

    We already have x -> f(x, a), but we want f(_, a), or even just f(a) if the first argument is implied.

We have this with macros

Both these conveniences are readily offered with, e.g., Chain.jl (as noted above).

using Chain
@chain "hello" split("") _.^2 join("•") uppercase
# …is the same as…
uppercase(join(split("hello", "").^2, "•"))

So why are we still talking about it? Well:

What @uniment’s MethodChains.jl proposal offers

The main selling point seems to be repurposing brace syntax { } to be function composition with built-in support for partial application.

There is a lot more to this proposal, but I’m not so sure about the rest of it.

Using braces as a Chain.jl syntax

We can achieve the core of this proposal simply by transforming the brace syntax into a Chain.jl-style chain:

using Chain
using MacroTools: postwalk

bracechains(expr) = postwalk(expr) do node
	if node isa Expr && node.head ∈ (:braces, :bracescat)
		arg = gensym()
		:($arg -> @chain $arg $(node.args...))
	else
		node
	end
end

To enable this syntax transformation in the REPL, run:

pushfirst!(Base.active_repl_backend.ast_transforms, bracechains)

Then you can do things like:

julia> "chains" |> {
           split("")
           _ .^ 2
           join("•")
           uppercase
       }
"CC•HH•AA•II•NN•SS"

julia> f = {repr, reverse, "(( $_ ))"}
#73 (generic function with 1 method)

julia> f(0xCAFE)
"(( efacx0 ))"

Differences

Personally, I find x |> {f, g} better than x.{f, g}, since to a Julian the former looks like function application while the latter looks like broadcasting.

I also prefer Chain.jl’s use of _ as the anonymous argument over it or any alphanumeric name.

Overall, this brace syntax is nice, but I’m not convinced it’s that much better than using Chain.jl as-is. Though nobody can deny it’s easier to type!

I think we should keep playing around like this… but not get too carried away! In the end, the simpler the better.

16 Likes

I can - typing { on my german keyboard requires pressing ALTGR (on the right side of the spacebar) as well as 7 (0 for }) in the number row (also on the right side of the keyboard). That’s more difficult to type than @ or |>, since those are on the left side of the keyboard, allowing me to use my left hand as well.

3 Likes

Quite creative, I like it!

I think there’s a certain nuance that’s missing though, which is:

Namely, I’m trying to create a concept which is as general as possible, and as a result, trying to avoid the behavior of threading the argument into a default position. Additionally, as has been expressed previously, the use of _ is somewhat wasteful, when within the local context it is possible to define new keywords—this would keep _ free for use in partial application, as PR #24990 proposes.

Indeed, if this proposal were to be accepted and #24990 were accepted (which is my hope), then many of the expressions here could look closer to Chain.jl:

"chains".{
    split(_, "")
    _.^2
    join(_, "•")
    uppercase
}

The other points that this misses are: 1) creation of an unneeded lambda (increases compile time) and 2) operator precedence. It’s frequent for chains to be short sub-expressions inside larger ones (e.g., arr.{filter(f, _), first}.a+1), and the lower precedence of |> forces more of it to be placed within the chain, e.g. arr |> {filter(f, _), first, it.a+1}, which is clumsier. The high precedence of . is useful.

Oh dear, now I understand the dislike for curly braces! :sweat_smile: How prevalent are such keyboards?

2 Likes

As far as I know, belgian and french too…

3 Likes

Another point I think bears re-emphasis:

A large motivation for pushing for acceptance as a language feature, is to get autocomplete support. Not only would this improve method discoverability as the OO folks have, but it would also make it so that threading into a default argument position (first for Chain.jl, last for DataPipes.jl) isn’t as compelling: for example, if this proposal and PR #24990 were accepted, then in "1:2:3".{split(_, ":")}, an autocomplete would likely fill out (_, ) and place the cursor after the comma, saving the effort of typing the underscore.

(Before I catch flak for the compile time of constructing a partial applicator, in proposal #2 I proposed giving _ preferred treatment within the chaining syntax so that it would be syntax-transformed into a simple function call. I didn’t restate that in this proposal, but I do still carry that intent. Perhaps I should code it in.)

As a result, I feel inclined to oppose the behavior of automatically threading into any default argument position, because the effort it saves would be negligible.

This is not good. How do they manage C-style languages? And in Julia, are they much less inclined to use type parameterization?

I suppose if we went forward with this proposal, an autocomplete would become popular very quickly: when typing obj., probably the first option to appear should be obj.{} so that these characters need not be typed. :sweat_smile: (of course, they’d be only entered if you hit <tab>.)

we suffer.

9 Likes

Well, it’s not that bad. For belgian, thumb and index at aligned (see link). But for french, indeed, that’s more painful …
Belgian keyboard layout
French keyboard layout
Bottom line, keep in mind the vast majority of languages use curly braces, and this doesn’t prevent these countries (Germany, France, Belgium, other …) to code in e.g. javascript / C / Go / etc. … :upside_down_face:

2 Likes

I did too, until I decided to do all coding with a US keyboard layout. It works well because in code files I write everything in English anyway. And every OS has a standard keyboard shortcut to switch layout for writing documents in French/German/etc. I’m much happier ever since, also having less strain in my fingers when coding.

3 Likes

[Off-topic? I.e. this post, as some others, only about typing on different keyboards, e.g. braces.]

That applies to e.g. the Icelandic keyboard too! For me at least it feel very natural to type in { and }. I suppose if people opposed very much Java, C and C++ and other curly-brace languages wouldn’t be popular (in Germany and some places)…

You do get people occasionally asking for this in Julia instead of begin … end. I’m very pro on not doing that (and it will not happen), not because of your objection, but it’s more useful for other things. Would this new syntax be most useful way of explaining those brackets?

Well @ for me is AltGr and Q so not easier to type with one hand… (I never understood why the other Alt no [allowed to] working, since it does nothing) as I type the braces. I suppose with two hands as I do slightly easier than the braces with one (or two hands, if I were to adjust to doing that).

Wow, that’s an awful (looking, e.g. for M) keyboard, with ( typed as, I suppose AltGR and, 5 (plus it being AZERTY, at least odd to me); or maybe not I guess you no longer for for using one hand (which was my first thought when responding, since I’m used to that).

1 Like

I agree with this summary. I would be curious to see the reception of the simplest parts of this, which is more or less “Chain.jl but with braces,” and with the differences you propose since I also prefer each of those, submitted as an actual PR into Base. It seems there is sometimes a different nature of discussion that happens on GitHub than on Discourse.

2 Likes

I like this approach very much … exepct the REPL customization part, which makes it non-easily reproducible / deployable. But it’s understandable due to its sheer experimental nature.

Building on top of that, I was considering that maybe a more Julian-consistent syntax could be

julia> "chains" pipe
           split("")
           _ .^ 2
           join("•")
           uppercase
       end

… where pipe (or similar) should be a reserved keyword, such as let, begin, do, etc.

1 Like

And it also doesn’t prevent us from developing carpal tunnel due to that :wink: We can’t change existing languages, but I’d rather not repeat the mistakes of the past when other solutions already exist.

A bit offtopic, but if you’ve ever wondered why Vim uses the ESC key for so many operations even though it’s very far out of the way - on the keyboard it was developed on, it was very accessible:


Also note the easy-to-press curly braces and pipe symbol (both left-shift+KEY (right-shift works too, but left-shift makes it once again easier with both hands)), which just aren’t an as prevalent thing on other keyboards…

2 Likes

If anything, the discussion over the last few comments has made me happier about the constraint to have the . dot in x.{f, g}, as it offers information for an autocomplete to fill in the curly braces {}. :sweat_smile:

That seems like an interesting idea. I would avoid it for short chains, e.g. arr.{first}.b[end], but for multi-line expressions it could be nice to have a more verbose syntax. This would be consistent with how the expression (a; b) is equivalent to the block expression begin a; b end.

I don’t know enough about parser design to know how difficult this would be to add to the language. But I like the idea.

2 Likes

In my opinion, not being worth it for small chains is an advantage. It would encourage people to just write

b(first(arr))[end]

plainly. Suddenly having a short, alternative spelling like a.{length} for every function call length(a) would inevitably lead to unfortunate bifurcation of style across the Julia codebase, if it were to become syntax.

Don’t let me shoot you down, though. I realise x.{f} is more amenable to autocompletion, etc. (And this objection doesn’t apply if it were just inside a macro.)

2 Likes

I suspect this might be one of the bigger hurdles: overcoming the notion that it’s simply “not Julian” to place the function name in suffix position instead of prefix position, or that accommodating two styles would be too confusing.

Maybe I’m just an optimist, but I don’t think it’s bad. I see it as similar to how we are free to say, “he measured the length of the baby” length(a) or equivalently “he measured the baby’s length” a.{length}. The latter might invoke an image of a man with a baby, pulling out a measuring tape; while the former might invoke an image of a man with a measuring tape, stumbling across a baby to measure. They could both be true at various instants, but the one we go with is whichever makes more sense in the context.

(and using the pipe operator a |> length is like saying, “he took the baby and fed it into a length measuring machine.” :laughing:)

Indeed, many things about this proposal would be easier if I simply settled on releasing a macro. :sweat_smile: My fear is that, if all I do is release a macro, then there is insufficient motivation for someone to develop an autocomplete that leverages it.

Regarding non-Julian things, I considered at the time to have @chain x { ... } but decided against it because it seemed bad to introduce a competitor block syntax to begin end in a macro I could see used widely across the ecosystem, just to save a few characters. Which was the same reason I called it @chain and not something more cryptic like @> or other options. I thought that would make it easier to search for as well if you didn’t know the macro.

12 Likes

Personally I think { ... } would be ideal to replace all the ... end blocks in Julia. begin end is way too long for common block macros like chain, but IMO its replacement could be general to all macros, rather than privileging the chain use case.