[ANN] Chevrons.jl: friendly >> chevron >> based data piping

Package Announcements
1

Announcing Chevrons.jl

I’ve made another package for chaining/piping data transformations together, similar to Chain.jl and Pipe.jl, stealing some of their ideas but with nicer >>-based syntax and REPL integration.

Here is a little example of using it with TidierData.jl:

julia> using Chevrons, DataFrames, TidierData

julia> Chevrons.enable_repl()  # magic to enable Chevrons syntax in the REPL

julia> df = DataFrame(name=["John", "Sally", "Roger"], age=[54, 34, 79], children=[0, 2, 4])
3×3 DataFrame
 Row │ name    age    children
     │ String  Int64  Int64
─────┼─────────────────────────
   1 │ John       54         0
   2 │ Sally      34         2
   3 │ Roger      79         4

julia> df >> @filter(age > 40) >> @select(num_children=children, age)
2×2 DataFrame
 Row │ num_children  age
     │ Int64         Int64
─────┼─────────────────────
   1 │            0     54
   2 │            4     79

What I particularly like about it:

  • Less repetititve: You don’t need to write @chain or @pipe every time, you can annotate an entire function, script or module with @chevrons to get the >> syntax.
  • REPL integration so you can use this syntax automatically.
  • >> is easier to type than |>.
  • Side-effects with >>>, similar to Chain.jl’s @aside, e.g. for logging intermediate values.

Quick comparison with similar packages:

Feature Chevrons.jl Chain.jl Pipe.jl
Piping syntax :heavy_check_mark: (>>) :heavy_check_mark: (@chain) :heavy_check_mark: (|>)
Side effects :heavy_check_mark: (>>>) :heavy_check_mark: (@aside) :x:
Pipe backwards :heavy_check_mark: (<<) :x: :x:
Recursive syntax :heavy_check_mark: :x: :x:
REPL integration :heavy_check_mark: :x: :x:
Line numbers on errors :x: :heavy_check_mark: :x:
21 Likes
2

Is this exclusive for DataFrames, or does this also work with general piping of stuff? How does this deal with integers?

3

Nope, it’s completely generic. Take a look at the README for more examples, such as:

julia> [5,2,4,3,1] >> filter!(isodd, _) >>> println("x = ", _) >> sum()
x = [5, 3, 1]
9
1 Like
4

If I’ve enabled the REPL feature, is there a way to escape >> if I intend to call the bit shift operator? e.g.

julia> 10 >> 1
5
2 Likes
5

Good question! You can do this by renaming the operator to something else and using it as an ordinary function:

julia> using Base: << as lshift

julia> lshift(1, 10)
1024

I have added this to the README thanks!

6 Likes
6

This is a nice package. I really hope Julia can support syntax like this.
However, I feel overwriting the << and >> operators could be problematic. Why not considering other operators? I think julia have a bunch of unused operators.

3 Likes
7

I agree that getting rid of bitshift << and >> would have meant that existing julia sourcecode could not work.

Please use another operators.

1 Like
8

You have to opt-in to this syntax with the @chevrons syntax, so it won’t break existing code. It’s purely a syntactic change to code annotated with @chevrons and won’t affect functions that that code calls.

5 Likes
9

Is there a way to use chevrons in a Pluto notebook without typing @chevrons in every cell?

3 Likes
10

Not currently. @fonsp does Pluto have any hooks that let us transform the AST of a cell before it is evaluated? (Would be cool to also have Chevrons.enable_pluto() or something.)

5 Likes