# \[RFC\] PipelessPipes.jl (now Chain.jl)

**URL:** <https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448>\
**Category:** Package Announcements\
**Created:** [November 19, 2020, 4:25pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448 "2020-11-19T16:25:38Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [November 23, 2020, 6:31am UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/21 "2020-11-23T06:31:20Z")

</div>

> [@jules](#):
>
> presumptive bugs

this one.

---

<div class="post-metadata">

**Author:** ![liuyxpp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/liuyxpp/32/9870_2.png) [@liuyxpp](https://discourse.julialang.org/u/liuyxpp)\
**Post date:** [November 23, 2020, 7:04am UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/22 "2020-11-23T07:04:45Z")

</div>

This is exactly what I want! I love it!

---

<div class="post-metadata">

**Author:** ![ajkeith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ajkeith/32/970_2.png) [@ajkeith](https://discourse.julialang.org/u/ajkeith)\
**Post date:** [November 23, 2020, 11:18am UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/23 "2020-11-23T11:18:00Z")

</div>

This looks great! I think the move from `@_` to `@chain` improves the clarity. Along the same lines, is `@!` set in stone? In Julia, I usually associate `!` with something mutating in place but the usage here feels pretty different to me. Maybe `@hop` , `@skip` , `@over` , or some other short descriptive word?

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [November 23, 2020, 11:48am UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/24 "2020-11-23T11:48:44Z")

</div>

Love the idea reminds me of R.

Weird question… would it be possible to emit intermediate values in say a let-block? Maybe something like: `@emit`

---

<div class="post-metadata">

**Author:** ![swt30](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/swt30/32/4667_2.png) [@swt30](https://discourse.julialang.org/u/swt30)\
**Post date:** [November 23, 2020, 12:54pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/25 "2020-11-23T12:54:57Z")

</div>

I’ve seen `tee` used for this elsewhere – e.g. at the shell, `tee` writes output to a file but also passes it on to the next step in the pipeline, and R has the ugly but useful “tee pipe” `%T>%` for the same thing. So maybe `@tee`.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 23, 2020, 1:16pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/26 "2020-11-23T13:16:21Z")

</div>

Nothing’s really set in stone, I’m always open for good arguments. I thought that `@!` was short and conveyed something like attention, unusual, out of the ordinary. I didn’t connect it with mutation, but if that seems confusing I could change to something like `@skip`. For me personally, `@tee` doesn’t convey any meaning, but I’m not an R user 🙂

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [November 23, 2020, 1:22pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/27 "2020-11-23T13:22:23Z")

</div>

`tee` is from the Linux/Posix command I would say. I thought also about `@aside`, but `@tee` should also be meaningful for a Linux user ?

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 23, 2020, 1:33pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/28 "2020-11-23T13:33:57Z")

</div>

> [@anon92994695](#):
>
> would it be possible to emit intermediate values in say a let-block? Maybe something like: `@emit`

I don’t quite understand the question, could you explain further?

---

<div class="post-metadata">

**Author:** ![ajkeith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ajkeith/32/970_2.png) [@ajkeith](https://discourse.julialang.org/u/ajkeith)\
**Post date:** [November 23, 2020, 1:35pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/29 "2020-11-23T13:35:40Z")

</div>

I’m also not familiar with the `tee` syntax but from a quick search it seems to imply writing to std out and/or files, which is not necessarily the case with the current `@!`.

I’m not sure if mutating was the right word choice here, I was referring to [this part](https://docs.julialang.org/en/v1/manual/style-guide/#bang-convention) of the style guide about functions like `push!` that modify their arguments. Given the popularity of `@.` for [broadcasting](https://docs.julialang.org/en/v1/manual/performance-tips/#More-dots:-Fuse-vectorized-operations) I think it might be common for people to expect `@!` to make an entire line modify arguments in some way. Either way, it’s a pretty minor syntax point in my opinion. I really like the overall feel!

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 23, 2020, 1:46pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/30 "2020-11-23T13:46:09Z")

</div>

I guess I hadn’t thought so much about a valid use case of _diverting_ the pipeline stream to some other target, like `tee` does. In that case it would make sense to choose a more descriptive word. I had envisioned `@!` more for temporary debugging. So the proposals so far are:

```julia
@skip
@hop
@over
@aside
@tee

```

Any other ideas? Given that you can do anything after this macro, not just divert into a different sink, I wonder if a more general word is better

---

<div class="post-metadata">

**Author:** ![pfitzseb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pfitzseb/32/45566_2.png) [@pfitzseb](https://discourse.julialang.org/u/pfitzseb)\
**Post date:** [November 23, 2020, 1:48pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/31 "2020-11-23T13:48:10Z")

</div>

`@without`? I also like `@skip`, fwiw.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 23, 2020, 1:52pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/32 "2020-11-23T13:52:54Z")

</div>

maybe `@also`? I wonder if `@skip` sounds a bit like, _skip that part altogether_

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [November 23, 2020, 1:57pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/33 "2020-11-23T13:57:20Z")

</div>

Yeah, I think `@skip` is a little confusing, but `@also` seems pretty good.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 23, 2020, 3:09pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/34 "2020-11-23T15:09:19Z")

</div>

or maybe `@btw` for _by the way_, although maybe not everybody knows that acronym

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [November 23, 2020, 3:50pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/35 "2020-11-23T15:50:37Z")

</div>

I would go for `@tee`: it’s really the same concept, and it’s well established. The name is also distinctive and short.

The picture on [tee (command) - Wikipedia](https://en.wikipedia.org/wiki/Tee_(command)) explains the origin of the name: the concept is to add a “T” in a pipeline, that makes the data available to two consumers.

It looks oddly specific to talk about files, but files are a very general concept in Unix. Here is a valid pipeline in Linux (Bash):

```bash
ls /usr |
tee >(grep ^lib | xargs du -hs >&2) |
grep ^s

```

In English:

```julia
Show the content of the /usr directory
Pass a copy to a sub-pipeline that keeps only entries starting with `lib` and prints their size to the second output
Pass the original data to `grep`, which keeps only entries starting with `s` and prints them to the first output

```

The output on my computer is:

```julia
sbin
share
src
7.7G	lib
17M lib32
4.0K	lib64
13M libexec
4.0K	libx32

```

The first three lines are from the main pipeline (first output). The last 6 lines are from the sub-pipeline (second output). (By default, both outputs are shown in the terminal.)

The trick in this example is `>(...)`, which creates a sub-pipeline that can be written to, like a file. Many things behave like “files” in Unix so `tee` is really about passing data to two consumers.

Isn’t this very similar to what we want here?

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [November 23, 2020, 3:54pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/36 "2020-11-23T15:54:53Z")

</div>

To me, `@tee` indicates something like you’re splitting the chain into two branches, each of which you can pass through a sequence of commands. But it seems like `@!` doesn’t give a whole subchain (in which you can chain outputs together) but rather a single command that doesn’t participate in the chain.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 23, 2020, 3:59pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/37 "2020-11-23T15:59:03Z")

</div>

I think you could actually do a whole chain, but at some point it might not be worth using `@chain` at all when it gets too complex…

In principle, this would work:

```julia
@chain 1:5 begin
    _ .^ 2
    @tee @chain begin
        sum
        println("The side result is $(_)")
    end
    diff
end

```

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 23, 2020, 4:01pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/38 "2020-11-23T16:01:36Z")

</div>

> [@sijo](#):
>
> the concept is to add a “T” in a pipeline

One could of course also just do `@T` if `tee` stands for that anyway

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [November 23, 2020, 4:15pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/39 "2020-11-23T16:15:31Z")

</div>

Yes, in complex cases I would rather write `@tee myfun(_)` and implement the second pipeline in `myfun`.

Regarding `@T`: indeed… I’m probably partial to `tee` because of my experience in shell programming. I find it more distinctive than `@T` (it’s a “real” word with a meaning). So it’s nice for people who already know `tee`. But it’s also nice the other way: learning `@tee` in Julia might help someone understand a shell pipeline one day 🙂

---

<div class="post-metadata">

**Author:** ![davide](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davide/32/52136_2.png) [@davide](https://discourse.julialang.org/u/davide)\
**Post date:** [November 23, 2020, 5:34pm UTC](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448/40 "2020-11-23T17:34:20Z")

</div>

I vote for `@aside`, it’s meaningful and the most clear and transparent to me.

[Previous page](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448.md?page=1)

[Next page](https://discourse.julialang.org/t/rfc-pipelesspipes-jl-now-chain-jl/50448.md?page=3)