# Yet another piping idea
**URL:** https://discourse.julialang.org/t/yet-another-piping-idea/37597
**Category:** Data
**Created:** [April 14, 2020, 11:16pm UTC](https://discourse.julialang.org/t/yet-another-piping-idea/37597 "2020-04-14T23:16:29Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [April 14, 2020, 11:16pm UTC](https://discourse.julialang.org/t/yet-another-piping-idea/37597/1 "2020-04-14T23:16:29Z")
With the upcoming DataFrames 1.0 release and the new `select` and `transform` functions, piping will become much idiomatic in basic DataFrames usage, i.e.
```julia
using DataFrames, Lazy
julia> @> df begin
transform(:b => (x -> x .+ 1) => :c)
select(:c)
end
```
One thing that I often want to do when doing this kind of piping in R is to stop in the middle of my chain and work in global scope for a bit and do something with a column I just created. But I don’t want to have to assign an intermediate dataframe. Lets say I have the following workflow
```julia
using DataFrames, Lazy
df = @> df begin
transform(:b => fun => :c)
end
v = do_something(df.c)
df = @> df begin
transform(:c => (x -> x .+ v) => :d)
end
```
ugh, what a pain! When with some macro magic we could do
```julia
df = @> df begin
transform(:b => fun => :c)
@stop _ # _ is the value returned from the pipe
v = do_something(_.c) # v now lives in whatever scope is outside this begin block
@continue # _ is the value returned from this loop.
transform(:c => (x -> x .+ v) => :d) # You can now access `v` here.
end
```
This may seem pretty excessive, but I can promise you it’s something I’ve wanted while using `dplyr`.
Just an idea I had, and thought I’d write it out to see if others have also wanted this.
---
### Author:  [@laborg](https://discourse.julialang.org/u/laborg)
#### Post date: [April 21, 2020, 4:31pm UTC](https://discourse.julialang.org/t/yet-another-piping-idea/37597/2 "2020-04-21T16:31:56Z")
I agree that that piping in dplyr is nice but but makes debugging or accessing intermediate results inconvenient.
Could a macro introduce a second level/type of `ans`? e.g.:
```julia
@> df transform(:b => fun => :c) # two exprs: feed first into second as first arg
@> select(:c) # one expr: feed last return of @> as first arg
@> df # one symbol: assign last return of @> to symbol
```
would be rewritten into:
```julia
_ANS = transform(df,:b => fun => :c)
_ANS = select(_ANS,:c)
df = _ANS
```
Grabing intermediate results might be as easy as having `@>>` return the last value without assigning it:
```julia
@> df transform(:b => fun => :c)
v = do_something(@>>) # v = do_something(_ANS); would (@>) be possible?
@> select(:c) # continue with _ANS
@> df
```
- It was always okay for me to repeat `%>%` at the end of a each line when working with dplyr; the repetition wasn’t visually bothering me - something I can’t say about the begin-end-blocks.
- I’ve no experience in writing macros so there might be some obvious problems with my proposal.
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [April 21, 2020, 5:50pm UTC](https://discourse.julialang.org/t/yet-another-piping-idea/37597/3 "2020-04-21T17:50:26Z")
There are ways better than hardcoding `_ANS` to make your idea work, one of them is `gensym()`, which makes a non-conflicting name.
I think a solution would probably look somewhat similar to that. Perhaps an array of expressions that are separated where `@stop` and `@continue` calls are. But this is beyond my abilities and time at the moment.
---
### Author:  [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)
#### Post date: [April 21, 2020, 8:27pm UTC](https://discourse.julialang.org/t/yet-another-piping-idea/37597/4 "2020-04-21T20:27:14Z")
> [@pdeffebach](#):
>
> One thing that I often want to do when doing this kind of piping in R is to stop in the middle of my chain and work in global scope for a bit and do something with a column I just created.
You actually can do this in R with the T-pipe from **magrittr** :
```r
> df <- data.frame(a = 1:2)
> df %>%
+ transform(b = 2 * a) %T>%
+ {v <<- 2 * .$b} %>%
+ transform(c = 3 * b)
a b c
1 1 2 6
2 2 4 12
> v
[1] 4 8
```