# Fixing the Piping/Chaining Issue

**URL:** <https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654>\
**Category:** Internals & Design\
**Tags:** proposal, piping, chaining, partial-evaluation, threading\
**Created:** [November 2, 2022, 11:04am UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654 "2022-11-02T11:04:36Z")\
**Posts on this page:** 20\
**Page:** 6

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [November 7, 2022, 10:39am UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/102 "2022-11-07T10:39:30Z")

</div>

> [@uniment](#):
>
> Yes, I believe this is the point. I don’t want any runtime computation whatsoever for something that just calls another function.

Literally the only reason the compiler was able to fold your computation was because the input to your `Fix` thing was part of the benchmarking expression. Here, let me help you out:

```julia
julia> @benchmark Fix{(1,2,-3,-1)}(f, vals, z=5)(args...; kw...) setup=(f=(args...;kwargs...)->(args...,(;kwargs...));vals=(:a,:b,:getting_close,:END); args=(:y, 1, 2); kw=(:k=>2,))
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max): 52.667 μs … 176.066 μs ┊ GC (min … max): 0.00% … 0.00%
 Time (median): 53.412 μs ┊ GC (median): 0.00%
 Time (mean ± σ): 55.087 μs ± 5.945 μs ┊ GC (mean ± σ): 0.00% ± 0.00%

  ▆█▅▂▁ ▃ ▁
  █████▆▅█▆▆▇▆▄▅█▆▅▆▆▄▄██▇▅▅▇▆▆▇▆▅▅▆▅▅▆▅▅▅▄▄▄▄▄▂▄▄▄▄▄▅▂▃▄▄▅▅▅▆ █
  52.7 μs Histogram: log(frequency) by time 84.9 μs <

 Memory estimate: 2.59 KiB, allocs estimate: 45.

julia> @btime Fix{(1,2,-3,-1)}(f, vals, z=5)(args...; kw...) setup=(f=(args...;kwargs...)->(args...,(;kwargs...));vals=(:a,:b,:getting_close,:END); args=(:y, 1, 2); kw=(:k=>2,))
  52.683 μs (45 allocations: 2.59 KiB)
(:a, :b, :y, 1, :getting_close, 2, :END, (k = 2, z = 5))

```

And ok, you might argue “but the function is fixed!”, then here, have this benchmark where only the input is not known and thus can’t be folded:

```julia
julia> @benchmark Fix{(1,2,-3,-1)}((args...;kwargs...)->(args...,(;kwargs...)), (:a,:b,:getting_close,:END), z=5)(args...; kw...) setup=(args=(:y, 1, 2); kw=(:k=>2,))
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max): 51.705 μs … 203.229 μs ┊ GC (min … max): 0.00% … 0.00%
 Time (median): 52.434 μs ┊ GC (median): 0.00%
 Time (mean ± σ): 54.838 μs ± 7.742 μs ┊ GC (mean ± σ): 0.00% ± 0.00%

  ██▄▂ ▁ ▁ ▂▂ ▁ ▂
  ███████▇▇█▁▇█▆▆█▅▅██▇▆█████▇▇██▇▇▇▇▇▆▆▆▇▆▇▆▆▆▆▅▆▆▆▇▇▆▆▅▆▆▆▄▅ █
  51.7 μs Histogram: log(frequency) by time 89 μs <

 Memory estimate: 2.55 KiB, allocs estimate: 44.

julia> @btime Fix{(1,2,-3,-1)}((args...;kwargs...)->(args...,(;kwargs...)), (:a,:b,:getting_close,:END), z=5)(args...; kw...) setup=(args=(:y, 1, 2); kw=(:k=>2,))
  51.934 μs (44 allocations: 2.55 KiB)
(:a, :b, :y, 1, :getting_close, 2, :END, (k = 2, z = 5))

```

It’s not better. The only case where it _could_ be better is if everything is tuple, always, which is just not a realistic use case.

To me your reply just shows that you’re ignorant about what you’re actually defining and how that actually works. This is not convincing.

> [@uniment](#):
>
> Moreover, and more importantly, it would allow more tightly specialized methods to float to the top of the list. When working with a distribution `d = Beta(1, 2)`, I should be able to type `d |>` and see that specialized methods such as `pdfsquaredL2norm`, which specialize on an argument `::Beta`, appear.

This again requires the autocomplete to know that `d` is a `Beta`, _which is information you don’t have_. Constructors are (sadly) not even required to return an object of their type, meaning you _have_ to run type inference, as I’ve repeated multiple times now, to even begin checking which methods can possibly receive `d`.

> [@uniment](#):
>
> In recognition that underscore placeholder syntax is good for chaining, but is actually sugar for partial function application, we should think through how to make it do that job well too. Because if we don’t, we will end up with _what was almost a solution_ to a bunch of other problems, but wasn’t quite good enough because we didn’t think it through. Maybe it’s just the engineer in me, but I have a bias toward trying to solve problems as generally as possible whenever I can. And, because,

Again, `_` has no relation whatsoever to the “no argument” case because you can’t pipe anything into functions that don’t take any arguments. There is no generalization possible here because the 0 argument case is not a generalization of the 1 argument case, and neither of the n-argument case.

> [@uniment](#):
>
> The opposite, actually. For example, `f=my_func(x, _...)` would allow me to call `f(a, b, c)` and it would execute `my_func(x, a, b, c)`. The way I’m proposing it, placeholder `_...` does not signify arguments you ignore, but vararg arguments you will fill in.

This already has syntax in the form of `f(d...) = my_func(x, d...)`, which is MUCH clearer about what’s going on than having to read the second part since `...` already has the established meanings.

> [@uniment](#):
>
> If the IDE can’t determine the type of the variable that you just typed because it’s part of an incomplete blob, and therefore cannot autocomplete, then how do the OOP guys get functioning autocomplete?

Because they are in a _statically typed_ language where the type of _every single object_ is determined just by having a method/function in the first place. In a statically typed language, not knowing the type of a variable is a compiler error. It is not in julia.

> [@uniment](#):
>
> Take the example above, `d = Beta(1, 2)`, a type from `Distributions.jl`. The object `d` has been declared to have type `Beta`, so when I type `d |>` the IDE should know that a) this is a `Beta` object

It cannot, because `Beta` at the syntax level is _just another function_ that can return an object of any type. Running at least type inference to know this is required.

> [@uniment](#):
>
> I’m about to pass it to a function (and most likely a partial function); so it should determine the available methods such as `kldivergence` and `invlogccdf` to show.

Whether or not the function was created with `_` or not has _no bearing on which methods are selected_. To the compiler, these two things are almost exactly the same, save for the name:

```julia
julia> f = (a,b) -> a+b
#3 (generic function with 1 method)

julia> g(a,b) = a+b
g (generic function with 1 method)

```

Whether `f` was created like `f = _ + _` or with the explicit anonymous function syntax is irrelevant - the compiler never even sees the difference.

> [@uniment](#):
>
> To me, it feels like the missing link to a respectable autocomplete is the ability to tell the IDE, in a way that is core to the language (and therefore worth the time and effort to develop around), what type of object I am going to call a method on before I begin looking for methods.

This already exists - annotate your types on every variable. Of course, this completely removes the ability of your code to be reused & generic, which is what you typically get in a static language. If you don’t want to do that and run type inference on every step, we AT LEAST require partial type inference to run up until invalid expressions, but again, that has NOTHING to do with whether you have fancy syntax for defining partially applied functions or not.

> [@uniment](#):
>
> Am I on the right track, or am I deluded? If I’m deluded, what is it that OOP languages have that Julia doesn’t, that allows their autocomplete to work?

I will just repeat myself from above: You want a specific syntax and ascribe to that syntax mythical abilities that have nothing to do whatsoever with the actual semantic problems underneath. OOP languages can have their easy autocomplete on their syntax because they are statically typed; not the other way around.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 7, 2022, 12:14pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/103 "2022-11-07T12:14:31Z")

</div>

> [@dlakelan](#):
>
> What’s the point of the whole Fix1 and FixLast baloney if you’re doing parsing changes?

First, even if you disagree with a proposal, please do not use language like that. Well-written and detailed proposals like this deserve to be heard and discussed in good faith, even if they are not implemented.

Simply lowering something without too much transformation is common in Julia. Consider eg `[]` (lowered as `hvcat` or variants, depending on content).

I think that lowering `\>` and `/>` with the proposed associative and precedence rules into a neutrally named generic function would be great. Then a package could take that and implement `FixFirst` and `FixLast` as described, or, if it pleases, do something completely different.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [November 7, 2022, 12:18pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/104 "2022-11-07T12:18:28Z")

</div>

> [@Sukera](#):
>
> Literally the only reason the compiler was able to fold your computation was because the input to your `Fix` thing was part of the benchmarking expression. Here, let me help you out:
> 
> …

We could do all the Julia benchmarks and include compile time if you prefer.

> [@Sukera](#):
>
> It cannot, because `Beta` at the syntax level is _just another function_ that can return an object of any type. Running at least type inference to know this is required.

Is running type inference problematic?

> [@Sukera](#):
>
> Whether or not the function was created with `_` or not has _no bearing on which methods are selected_.

Never said it did.

> [@Sukera](#):
>
> that has NOTHING to do with whether you have fancy syntax for defining partially applied functions or not.

The first bit of information you need, before you can begin your search for methods that dispatch on your object, is a) what the object is, and b) the fact that you are about to call a function on it. That’s what piping + fancy partial application syntax provide, in an order that’s convenient (and therefore accessible and likely to be used) for the human-machine interaction in question.

I’m not claiming to solve autocomplete, not by a long shot. I’m just hoping to get over one of the first hurdles so that it can be in reaching distance.

> [@Sukera](#):
>
> Because they are in a _statically typed_ language where the type of _every single object_ is determined just by having a method/function in the first place.
> 
> …
> 
> OOP languages can have their easy autocomplete on their syntax because they are statically typed

Perhaps I shall inform [this person](https://discourse.julialang.org/t/my-mental-load-using-julia-is-much-higher-than-e-g-in-python-how-to-reduce-it/18902) that they are not allowed to use autocomplete in Python, because it’s not statically typed.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 7, 2022, 12:58pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/105 "2022-11-07T12:58:48Z")

</div>

> [@uniment](#):
>
> I’m not claiming to solve autocomplete, not by a long shot. I’m just hoping to get over one of the first hurdles so that it can be in reaching distance.

This (long) discussion now features

1. a proposed syntax,
2. claims about the necessity and feasibility of autocompletion,
3. various side-discussions about performance and implementation issues.

Personally, I think that the interesting part is 1., and 2. and 3. are distractions and sidetrack the readers from something that would be great to add to the language in its essential form (a pair of operators with the proposed syntax and precedence, that lower into a generic function, which packages could then define methods for).

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [November 7, 2022, 1:06pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/106 "2022-11-07T13:06:52Z")

</div>

Welcome back!

Sorry, but you missed that I jumped ship, and am now in support of essentially the proposal of #24990 ([see reasoning here](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/74)), with proposal for a truly generalized `Fix` type ([description here](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/98)). Still a rough draft.

In short, if done right, underscore placeholder syntax can lower into typed partial application functors, and more generally (and legibly), while treating the parser with greater kindness, than my original proposal for `\>` and `/>`.

I hope it doesn’t disappoint 🤞

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [November 7, 2022, 1:35pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/107 "2022-11-07T13:35:06Z")

</div>

> [@uniment](#):
>
> Is running type inference problematic?

Not necessarily, but right now there is no capability to run it on broken/incomplete expressions. That has to be built first, after which you’re still left with the question of what to do in functions that don’t have their types annotated, where type inference won’t help in the slightest because it’ll end up as giving you every possible function the IDE could know about.

> [@uniment](#):
>
> The first bit of information you need, before you can begin your search for methods that dispatch on your object, is a) what the object is, and b) the fact that you are about to call a function on it. That’s what piping + fancy partial application syntax provide, in an order that’s convenient (and therefore accessible and likely to be used) for the human-machine interaction in question.

And I’m arguing that no, they don’t solve either problem. You need (at least partial) type inference to know what type an object is. Once you’ve created an object, _of course_ you’re going to pass it to a function (or `return` it, but I’ll assume that this is not wanted), what else are you going to do? Even operators are just functions in julia. Pretty much anything can be made callable and is thus potentially the thing to use the object on.

> [@uniment](#):
>
> I’m not claiming to solve autocomplete, not by a long shot. I’m just hoping to get over one of the first hurdles so that it can be in reaching distance.

That’s ok - but do see that the surface level syntax is not the “first hurdle” to overcome and that it doesn’t solve the fundamental problem of “there is not enough information available to the IDE to autocomplete with”.

In fact, if you start writing a function name, editors using LanguageServer.jl are already suggesting potential names, so autocomplete in that regard already works - even without type inference. It does need that additional context of “here’s the first few characters” to filter out potential matches, but that’s already how autocomplete in other languages works anyway.

> [@uniment](#):
>
> Perhaps I shall inform [this person](https://discourse.julialang.org/t/my-mental-load-using-julia-is-much-higher-than-e-g-in-python-how-to-reduce-it/18902) that they are not allowed to use autocomplete in Python, because it’s not statically typed.

Polemic rhetoric aside, you could also just ask them how they continue to develop one of the big plotting packages in julia, [Makie.jl](https://github.com/MakieOrg/Makie.jl), if the autocomplete is so unbearable 🤷 They’re a main contributor after all and even published [a paper](https://joss.theoj.org/papers/10.21105/joss.03349) about it.

That post is also 4(!) years old by now and a lot has changed since then. Atom/Juno was still widely used. There was no VSCode extension. [LanguageServer.jl](https://github.com/julia-vscode/LanguageServer.jl) was barely beginning to start to use the very first versions of [SymbolServer.jl](https://github.com/julia-vscode/SymbolServer.jl), which is what’s used right now for providing the symbols for autocompletion in the first place. I don’t think the thread is representative of the problems faced today.

In regards to python’s autocomplete - try this and tell me what you get:

```julia
[sukera@tempman ~]$ python3
Python 3.10.8 (main, Oct 13 2022, 21:13:48) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo:
... x = 1
... 
>>> Foo().<TAB>

```

or this:

```julia
[sukera@tempman pytest]$ python3
Python 3.10.8 (main, Oct 13 2022, 21:13:48) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo:
... def test(self):
... return 1
... 
>>> Foo().tes<TAB>

```

I can press `<TAB>` as many times as I like, I can’t get it to autocomplete. I can get that to work in `ipython`, mind you, but the exact same also already works in our REPL. What I can’t get to work even in `ipython` is then writing a function:

```julia
def baz(x):
    x.<TAB>

```

and it won’t autocomplete. How could it? It has no idea what type `x` might be. This is the exact problem I’m trying to convey is hard. It will do it if I write `def baz(x: Foo):`, because then it knows the type again, which is exactly the behavior I see in julia with e.g. VSCode and LanguageServer.jl once it knows the type - just like in any other statically typed language. Annotating the type however removes exactly that genericity we so desire for dynamic workflows.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 7, 2022, 2:55pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/108 "2022-11-07T14:55:48Z")

</div>

> [@Tamas\_Papp](#):
>
> please do not use language like that.

Was not intending anything, just a poor choice of words, perhaps “rigamarole” closer to what I meant.

I think the proposal to have a bunch of currying capacity is a good idea, I just think the piping situation is basically a syntactic transform and should be attacked at that level.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 7, 2022, 3:13pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/109 "2022-11-07T15:13:20Z")

</div>

> [@Sukera](#):
>
> ```julia
> def baz(x):
> x.<TAB>
> 
> ```
> 
> and it won’t autocomplete. How could it? It has no idea what type `x` might be.

Indeed, I’m glad to see someone bring this up, the assertion that somehow piping is going to improve autocomplete seems a bit optimistic to put it mildly.

In an IDE like vscode if you type something like

```julia
foo |> TAB

```

Under what circumstances does it gain any information over just

```julia
TAB

```

only when it knows the type of foo and can search through all the methods of all the functions that are specialized for that type OR take a generic type for the first argument. The number of functions that take a generic type for a first argument will be very high because of generic duck typing in Julia so it isn’t going to be super helpful. You’ll wind up with 26000 options often enough.

Furthermore there will be many if not most cases where the type of foo can’t be figured out. Such as

```julia
function bar(foo, baz)
   foo |> TAB

```

What it might do is help with top level scripts where global vars are being used. But if you care in any way about performance you’re still writing function barriers for top level scripts. Oddly the function barrier helps the compiler but makes the IDE have no idea what’s up

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [November 7, 2022, 3:23pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/110 "2022-11-07T15:23:52Z")

</div>

@Sukera If I understand your concern correctly, you are saying that adding a syntax like this does not fundamentally solve the problem of autocomplete since it’s still necessary to do type inference. That makes sense to me. However, it does seem plausible to me that syntax like this could make for a more convenient autocomplete UI experience at the user level, conditional on further engineering work under the hood—would you agree with that?

I think multiple contributors to the discussion have expressed what is also my sentiment very well, that the original proposal seems like a surprisingly elegant pair of operators, purely for syntactic convenience, and does not have to necessarily have the full power of being a perfectly consistent currying functor etc. etc.

They would not be the first operators to bind more tightly than function calls; both type annotation `::` and broadcast `.` and `do` keyword do so I believe.

> [@dlakelan](#):
>
> I think the proposal to have a bunch of currying capacity is a good idea, I just think the piping situation is basically a syntactic transform and should be attacked at that level.

> [@bertschi](#):
>
> A similar idea is proposed here: `/>` and `\>` bind more strongly than function application. It is correct though, that no such operators currently exist in Julia. In a sense all infix operators are syntactic sugar

> [@merlin](#):
>
> in data engineering or anlysis work. IMO this is a useful extension of the pipe idiom

> [@MattEri](#):
>
> No idea if it’s viable in Julia or not, or if it’s worth the trouble, but it would probably make a lot of the functional style stuff less cumbersome to code and parse mentally.

> [@Tamas\_Papp](#):
>
> I think that lowering `\>` and `/>` with the proposed associative and precedence rules into a neutrally named generic function would be great.

So, it seems there are many who would be excited for syntactic sugar for FrontFix and BackFix (or FrontPipe and BackPipe ? )

Most of the concerns I’m seeing seem like they revolve around either 1. some of the desired power of the operator or 2. some of the claimed second-order benefits of the operator, when neither of these points are actually the key selling feature, which is a nice way to thread objects through some common functional patterns.

What if the transformation is the following? Using the same `$` as function application notation:

`any_expr /> any_callable $ {args...}` becomes `any_callable $ {any_expr, args...}`

And `any_expr /> any_callable` becomes `args -> any_callable $ {any_expr, args...}`. And vice versa for backfix. Thus for one-argument functions `foo`, then `x /> foo() == x |> foo == x \> foo()`

Using the example that breaks the powerful bells-and-whistles version from @CameronBieganek  
`[4, 9, 16] \> map $ {sqrt} \> filter $ {iseven}`

We transform each `\>` right-associatively, so

`([4, 9, 16] \> map $ {sqrt}) \> filter $ {iseven}`

then

`filter $ {iseven, ([4, 9, 16] \> map $ {sqrt})}`

then inner transformation

`filter $ {iseven, map $ {sqrt, [4, 9, 16]}} #evaluates to [2, 4]`

So it looks like treating the operations just as syntax transformation no longer breaks. Did I do this correctly?

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [November 7, 2022, 3:43pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/111 "2022-11-07T15:43:28Z")

</div>

> [@adienes](#):
>
> However, it does seem plausible to me that syntax like this could make for a more convenient autocomplete UI experience at the user level, conditional on further engineering work under the hood—would you agree with that?

No, I do not. `\>` or `/>` only exclude functions that don’t take any arguments at all from being possible matches, which aren’t considered for piping in the first place - they don’t take arguments after all. The potential engineering work by itself can inform possible method completions, but those are not going to be improved/further culled by knowing that one of the arguments to be piped is of a given type (and if it can, only in _very_ limited cases, where the first/last argument is not already of the same type in all methods and dispatch is used for disambiguation).

> [@adienes](#):
>
> What if the transformation is the following?

The trouble with your proposal is that I can make any object callable, and that is not information available just from parsing:

```julia
julia> struct FooBar end

julia> (f::FooBar)(x) = "A `FooBar` instance got called with argument $(x)!"

julia> foobar = FooBar()
FooBar()

julia> foobar(42)
"A `FooBar` instance got called with argument 42!"

```

So to resolve that, you already need to know the boundary of where to stop resolving/applying the call you want to move around, which is the most contentious point of the `_` proposal (julia will happily parse any symbol followed directly by parentheses as a `Expr(:call, ...)`, but that has no bearing on whether or not the call will succeed. You either need to know ahead of time with some other information or pick semantics of when to stop moving calls around).

Another problem is that composing that with `|>` (which we can’t remove, would be breaking) is pretty hard - you can’t easily change lowering for it because it already has defined semantics:

```julia
julia> Meta.parse("a |> b |> c |> d")
:(((a |> b) |> c) |> d)

julia> Meta.parse("a |> b |> c |> d") |> dump
Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: Symbol |>
    2: Expr
      head: Symbol call
      args: Array{Any}((3,))
        1: Symbol |>
        2: Expr
          head: Symbol call
          args: Array{Any}((3,))
            1: Symbol |>
            2: Symbol a
            3: Symbol b
        3: Symbol c
    3: Symbol d

```

> [@adienes](#):
>
> Using the same `$` as function application notation:

> [@adienes](#):
>
> So it looks like treating the operations just as syntax transformation no longer breaks. Did I do this correctly?

Possibly? The problem is that we don’t have any notion of “function application operator” and thinking of `foo()` as doing that is not correct, as (I think) has been pointed out either somewhere far above or in the other thread.

If you want to see it as just a syntax transform, the best way to test your idea out is to write a macro to do it, since that’s exactly what macros already do. Whether or not such a transform is wanted/useful in Base is then a different discussion, unrelated to whether the idea works in principle.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [November 7, 2022, 5:13pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/112 "2022-11-07T17:13:15Z")

</div>

> [@Sukera](#):
>
> Possibly? The problem is that we don’t have any notion of “function application operator” and thinking of `foo()` as doing that is not correct, as (I think) has been pointed out either somewhere far above or in the other thread.

This is probably my own ignorance, but I admit I don’t understand why this is more difficult than `do`-block parsing.

can `A(w) /> B(x) /> C(y) /> D(z)` parse to something like

```julia
D(z) do
   C(z) do
        B(x) do
            A(w)
        end
    end
end

```

Although I know that `do`-block syntax wants anonymous functions passed into first argument not arbitrary expressions.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 7, 2022, 5:52pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/113 "2022-11-07T17:52:20Z")

</div>

> [@adienes](#):
>
> parse to something like

it would parse to `A(w) /> B(x) /> C(y) /> D(z)` as it should, just like do parses to a do block:

```julia
julia> Meta.parse("map([1,2,3]) do x ; x+1 end")
:(map([1, 2, 3]) do x
      #= none:1 =#
      x + 1
  end)

```

The question ultimately is can it mean something other than application of functions.

So basically these proposals are proposals to make operators work like operators (ie. lower to calls to functions) **EXCEPT** if the operator is `\>` or `/>` in which case it should specially lower to something else. This is very similar to introducing a “special form” such as “for” or “let” or “begin” or “do” except the special form would **look like** operators.

In some sense, `for` and such are exactly the same, they lower to stuff that doesn’t look anything like a for loop:

```julia
julia> f(x) = for i in 1:10 ; println(i); end
f (generic function with 1 method)

julia> @code_lowered(f(1))
CodeInfo(
1 ─ %1 = 1:10
│ @_3 = Base.iterate(%1)
│ %3 = @_3 === nothing
│ %4 = Base.not_int(%3)
└── goto #4 if not %4
2 ┄ %6 = @_3
│ i = Core.getfield(%6, 1)
│ %8 = Core.getfield(%6, 2)
│ Main.println(i)
│ @_3 = Base.iterate(%1, %8)
│ %11 = @_3 === nothing
│ %12 = Base.not_int(%11)
└── goto #4 if not %12
3 ─ goto #2
4 ┄ return nothing
)

```

In general I think it’s a **very very bad idea** to introduce a “special form” that looks like function / operator application. One of the absolutely best things about Julia is that whenever you look at the code you know exactly what the hell is going on. One of the absolute worst things about R is that absolutely anywhere you could run into something that looks like a function call but is actually “nonstandard evaluation”.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [November 7, 2022, 5:57pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/114 "2022-11-07T17:57:59Z")

</div>

> [@dlakelan](#):
>
> In general I think it’s a **very very bad idea** to introduce a “special form” that looks like function / operator application

Fair enough, then what if they just use different keywords so it looks less like a function? e.g. `into` and `over` for front/backfix

`[4, 9, 16] over map(sqrt) over filter(iseven) # evalutes to [2, 4]`

Then syntax highlighting makes it purple and it’s clear it’s nonstandard evaluation. I anticipate most people would care more about the functionality than the exact semantics of expression

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 7, 2022, 6:02pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/115 "2022-11-07T18:02:35Z")

</div>

> [@adienes](#):
>
> `[4, 9, 16] over map(sqrt) over filter(iseven) # evalutes to [2, 4]`

as opposed to

```julia
filter(iseven,map(sqrt,[4,9,16]))

```

or something like:

```julia
[4,9,16] |> ComposeChain(Curry(map,sqrt),Curry(filter, iseven))

```

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [November 7, 2022, 6:09pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/117 "2022-11-07T18:09:55Z")

</div>

@dlakelan I can only speak for myself, but I would prefer a threading syntax over any of those other options. I usually default to Chain.jl in these situations.

```julia
@chain [4, 9, 16] begin
    map(sqrt, _)
    filter(iseven, _)
end

```

But if `into / over`-like syntax existed I would probably use that.

I very much enjoy thinking functionally, but when I type something like this  
`filter(iseven,map(sqrt,[4,9,16]))`

I will quite literally type it backwards right-to-left starting with the innermost function call. It is not super ergonomic but it’s how my brain works haha. I have full conviction that somewhere buried in this (long) thread we have the pieces to build a useful piece of syntax to make these patterns more ergonomic.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 7, 2022, 6:13pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/118 "2022-11-07T18:13:24Z")

</div>

> [@adienes](#):
>
> I usually default to Chain.jl in these situations.

Indeed, I really like Chain and it’s imported by DataFramesMeta.jl so I wind up with it by default almost everywhere I’d want it.

But if we want **syntax** then I think the right thing to do is propose an alternative macro:

```julia
@piperator [4,9,16] |> map(sqrt,_) |> dothing(_,[1,2,3]) |> somesuch(a,_,b)

```

In fact I think we already have Pipe.jl that does this right?

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 7, 2022, 6:16pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/119 "2022-11-07T18:16:47Z")

</div>

and if you want a curried function, how about

```julia
@currierandives f(_,b) |> q(a,_,c) |> filter(bar,vcat(b,_))

```

Which results via syntax transformation in something like:

```julia
x-> filter(bar,vcat(b,q(a,f(x,b),c))

```

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [November 7, 2022, 6:20pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/120 "2022-11-07T18:20:51Z")

</div>

> [@dlakelan](#):
>
> In fact I think we already have Pipe.jl that does this right?

I like Chain.jl a lot, and I have nothing against Pipe.jl. But I can’t help but come back to one of @uniment 's opening statements:

> [@uniment](#):
>
> I’ve counted a dozen packages that implement chaining in some form or other. For such a common and generic idiom, maybe leaning on the package ecosystem for this is the wrong approach.

Wouldn’t it be nice to type

```julia
x = mystruct into foo(arg1, arg2) into bar(arg3, arg4)

```

As a _built-in_ ? Even using these macros we need to remember to include it everywhere, it’s still a few more characters to type, we need to trust that they will continue to be maintained as necessary, and other packages will not necessarily be designed to interop smoothly by default with whatever syntax is chosen by these third-party macros.

Obviously the following comparison is extreme maybe to the point of uselessness, but imagine if all we had were `while` loops and you had to use a third-party macro to run a `for` loop. This is somewhat how I feel about the current underpowered state of currying/piping.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [November 7, 2022, 6:21pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/121 "2022-11-07T18:21:31Z")

</div>

> [@Sukera](#):
>
> Polemic rhetoric aside, you could also just ask them how they continue to develop one of the big plotting packages in julia, [Makie.jl](https://github.com/MakieOrg/Makie.jl), if the autocomplete is so unbearable 🤷 They’re a main contributor after all and even published [a paper](https://joss.theoj.org/papers/10.21105/joss.03349) about it.

That’s quite impressive. Perhaps they can update their original question with the answer, as I have yet not found it.

> [@Sukera](#):
>
> I can get that to work in `ipython`, mind you, but the exact same also already works in our REPL.

Let’s see if we can do with Julia what we can do with Python…

**Python:**

Let’s define a class:

```python
class Foo:
    x = 1
    def bar(self):
        return self.x

```

Tab-complete shows the property `x` as well as the member method `bar`.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/d/4/d4b02c6dd3052eb1d69596e8675cfa05682eb088.png)

**Julia:**

The Julian way of writing this is to recognize that `Foo`’s member method `bar` probably generalizes across a range of other `Foo`-like types, so we define an `AbstractFoo` type and specialize our methods on it. (We could specialize to `Foo`, and that would do exactly the same thing as having a class member method, but that’s not as Julian.) First we write `Foo`-specialized methods which directly access its fields, and then write methods of `AbstractFoo` to call them.

```julia
abstract type AbstractFoo end
Base.@kwdef struct Foo <: AbstractFoo
    x = 1
end
getx(foo::Foo) = foo.x
bar(foo::AbstractFoo) = getx(foo)

```

 ![image](https://global.discourse-cdn.com/julialang/original/3X/8/6/86bc8ba1f49709373373128a59731c5b913deaf9.png)

Because we have syntax sugar for `getproperty` (namely `.` dot), and because I have typed `Foo()`, it knows to call `propertynames` on this object (or something similar). Because I have typed the object description, and typed a dot, autocomplete has the information it needs to help me discover `x`.

However, it’s not Julian to access the object’s properties directly; the preferred idiom is instead to call methods on it. So let’s find those methods.

Unfortunately, the situation isn’t so good for helping me discover either `bar(::Foo)`, or `getx(::Foo)`:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/2/f/2fcb4e4fa0377c64c7f311da924a7e6e70e32b1d.png)

What I claim is simple: that one day, autocomplete will recognize that I intend to call a function that specializes on a `Foo`, and it will help me find it. However, because the pipe operator can only call single-argument functions, at the moment such an autocomplete would not be very useful; we need a preferred partial application technique first, so that we can capture the whole range of possible method signatures.

Unless, of course, we should just solve the problem by making every method a dotted member method:

```julia
abstract type AbstractFoo end
Base.@kwdef struct Foo <: AbstractFoo
    x = 1
    bar = function(self) self.x end
end
Base.getproperty(x::AbstractFoo, n::Symbol) = begin
    if getfield(x,n) isa Function
        return (a...;k...)->getfield(x,n)(x,a...;k...)
    end
    getfield(x,n)
end

```

Now we finally get method discoverability in Julia, but it shouldn’t be this hard (or entirely non-Julian) to do.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/b/e/bea0d1da82c9e776a9a9fb591002e18e6e611c4a.png)

> [@Sukera](#):
>
> What I can’t get to work even in `ipython` is then writing a function:
> 
> ```julia
> def baz(x):
> x.<TAB>
> 
> ```
> 
> and it won’t autocomplete. How could it? It has no idea what type `x` might be. This is the exact problem I’m trying to convey is hard.

That’s not the problem I care about, because I do not care about solving impossible problems. I would have no time left for living life if I did.

> [@dlakelan](#):
>
> The number of functions that take a generic type for a first argument will be very high because of generic duck typing in Julia so it isn’t going to be super helpful. You’ll wind up with 26000 options often enough.

You can consider a “member method” to be simply a function which is specialized to _exactly the concrete type_ of the class. When hoping to tab-complete on member methods, I’m trying to find the most specialized functions! Why should I wish to hit tab to see all the methods which are _not_ specialized to this object, nor to any object like it? I can just start typing random function names anyway.

> [@dlakelan](#):
>
> What it might do is help with top level scripts where global vars are being used.

Recall from inception the purpose of Julia, which is essentially to be a scripting language which compiles: to have the benefits of quick dev time and then quick run time. Scripting languages are quick and easy to develop in, partly because of not needing to assign types to objects, sure ok, but also because if you want to scratchpad something you can just casually start scratchpadding. You can hop into a REPL and experiment, and when you’ve found the appropriate methods, algorithms, and fragments of glue code you can copypaste back into the editor; or you can hit CTRL+ENTER in VSCode, or etc. It’s part of the preferred development style.

For sections of code where arguments are fully generic to `::Any`, and if you’re also not scratchpadding the code and getting global vars from it, I don’t see why anyone would expect autocomplete to help.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 7, 2022, 6:28pm UTC](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654/122 "2022-11-07T18:28:21Z")

</div>

> [@adienes](#):
>
> trust that they will continue to be maintained as necessary, and other packages will not necessarily be designed to interop smoothly by default with whatever syntax is chosen by these third-party macros.

I think these are two separate questions. If this is really a concern, then lobbying to get Chain.jl and Pipe.jl into Base seems reasonable. At the moment I don’t think there’s a real concern here. Both Chain.jl and Pipe.jl are under some kind of MIT license, it would always be possible for Julia Base or the julialang organization to absorb them in the future.

> [@adienes](#):
>
> Wouldn’t it be nice to type
> 
> `x = mystruct into foo(arg1, arg2) into bar(arg3, arg4)`

IMHO no, why do we need special syntax for this when macros are perfect for special syntax? and the \_ syntax is actually more understandable imho. and it’s going to be annoying as hell to write macros that work with this syntax, also now you will never be able to use `into` or `over` as variable names and so it’ll break people’s code who have used those variable names. Again I think it’s “julia is no longer at that stage” territory.

[Previous page](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654.md?page=5)

[Next page](https://discourse.julialang.org/t/fixing-the-piping-chaining-issue/89654.md?page=7)