Fixing the Piping/Chaining Issue

Note that the work of creating the @currierandives macro is about 90% done by Pipe.jl you just need to add an anonymous function creation to the output:

julia> using Pipe

julia> @macroexpand @pipe f(_,b) |> q(a,_,c) |> filter(bar,vcat(b,_))
:(filter(bar, vcat(b, q(a, f(_, b), c))))

@adienes What do you think?

I am leaning to support #24990, with the minor modification of supporting _... for varargs slurping to improve the underscore syntax’s generality.

@adienes It sounds like what you are proposing is basically “front pipe” and “back pipe”, implemented at the syntax level. I actually proposed that in the currying PR a couple years ago. :slight_smile:

It would be possible to have a syntax level front pipe \> and back pipe \>> such the parser translates

a \> f(b, c)

into

f(a, b, c)

and translates

c \>> f(a, b)

into

f(a, b, c)

The front and back pipe “syntactic operators” (I just made up that term) would be parsed with left associativity, so

a \> f(b, c) \> g(d, e)

would be equivalent to

(a \> f(b, c)) \> g(d, e)

which the parser would translate to

g(f(a, b, c), d, e)

But of course there are some weaknesses to this approach. As far as I know, if you only have syntactic front and back pipes, there’s no easy way to express something like this:

b |> f(a, _, c) # currying PR
3 Likes

Ah so this is why you were so critical of my original proposal; you had already been a proponent of something nearly identical :wink: I don’t know though, underscore placeholder syntax is growing on me.

I’m currently trying to imagine how to express combining multiple objects, something like…

([1, 2, 3], ", ") |> join(_, _)

I know this doesn’t work; I’m trying to think of what the syntax could look like.

3 Likes

Simple:

([1, 2, 3], ", ") |> Splat(join(_, _))
help?> Splat
search: Splat splitpath display displaysize displayable redisplay

  Splat(f)

  Equivalent to

      my_splat(f) = args->f(args...)

  i.e. given a function returns a new function that takes one argument
  and splats its argument into the original function. This is useful as
  an adaptor to pass a multi-argument function in a context that expects
  a single argument, but passes a tuple as that single argument.
  Additionally has pretty printing.

  │ Julia 1.9
  │
  │  This function was introduced in Julia 1.9, replacing
  │  Base.splat(f).
5 Likes

Thanks for this! I’m not a fan of the flow involved with ?(x, y<tab>, so it’d be neat if this could be an autocomplete option when constructing Tuples of arguments… or maybe FrankenTuples…

Just a quick confirmation that the new Fix functor operates as expected, running @code_llvm confirms it still compiles to the same thing.

julia> f() = map([1,2,3]) do x x+1 ; end
f (generic function with 1 method)

julia> g() = map(x->x+1,[1,2,3])
g (generic function with 1 method)

julia> h() = FixLast(map, [1,2,3])(x->x+1)
h (generic function with 1 method)

julia> i() = Fix{(2,)}(map, ([1,2,3],))(x->x+1)
i (generic function with 1 method)

result

julia> @code_llvm f()
;  @ REPL[17]:1 within `f`
; Function Attrs: uwtable
define nonnull {}* @julia_f_2255() #0 {
pass.2:
  %gcframe9 = alloca [4 x {}*], align 16
  %gcframe9.sub = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 0
  %0 = bitcast [4 x {}*]* %gcframe9 to i8*
  call void @llvm.memset.p0i8.i32(i8* noundef nonnull align 16 dereferenceable(32) %0, i8 0, i32 32, i1 false)
  %1 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 2
  %2 = bitcast {}** %1 to { {}* }*
  %3 = call {}*** inttoptr (i64 1699154720 to {}*** ()*)() #3
; ┌ @ array.jl:126 within `vect`
; │┌ @ array.jl:679 within `_array_for` @ array.jl:676
; ││┌ @ abstractarray.jl:840 within `similar` @ abstractarray.jl:841
; │││┌ @ boot.jl:468 within `Array` @ boot.jl:459
      %4 = bitcast [4 x {}*]* %gcframe9 to i64*
      store i64 8, i64* %4, align 16
      %5 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 1
      %6 = bitcast {}** %5 to {}***
      %7 = load {}**, {}*** %3, align 8
      store {}** %7, {}*** %6, align 8
      %8 = bitcast {}*** %3 to {}***
      store {}** %gcframe9.sub, {}*** %8, align 8
      %9 = call nonnull {}* inttoptr (i64 1698961392 to {}* ({}*, i64)*)({}* inttoptr (i64 294838384 to {}*), i64 3)
      %10 = bitcast {}* %9 to i64**
      %11 = load i64*, i64** %10, align 8
; │└└└
; │┌ @ array.jl:966 within `setindex!`
    %12 = bitcast i64* %11 to <2 x i64>*
    store <2 x i64> <i64 1, i64 2>, <2 x i64>* %12, align 8
    %13 = getelementptr inbounds i64, i64* %11, i64 2
    store i64 3, i64* %13, align 8
; └└
; ┌ @ abstractarray.jl:2933 within `map`
; │┌ @ array.jl:716 within `collect_similar`
    store {}* %9, {}** %1, align 16
    %14 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 3
    store {}* %9, {}** %14, align 8
    %15 = call nonnull {}* @j__collect_2257({}* nonnull %9, { {}* }* nocapture readonly %2) #0
    %16 = load {}*, {}** %5, align 8
    %17 = bitcast {}*** %3 to {}**
    store {}* %16, {}** %17, align 8
; └└
  ret {}* %15
}

julia> @code_llvm g()
;  @ REPL[18]:1 within `g`
; Function Attrs: uwtable
define nonnull {}* @julia_g_2258() #0 {
pass.2:
  %gcframe9 = alloca [4 x {}*], align 16
  %gcframe9.sub = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 0
  %0 = bitcast [4 x {}*]* %gcframe9 to i8*
  call void @llvm.memset.p0i8.i32(i8* noundef nonnull align 16 dereferenceable(32) %0, i8 0, i32 32, i1 false)
  %1 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 2
  %2 = bitcast {}** %1 to { {}* }*
  %3 = call {}*** inttoptr (i64 1699154720 to {}*** ()*)() #3
; ┌ @ array.jl:126 within `vect`
; │┌ @ array.jl:679 within `_array_for` @ array.jl:676
; ││┌ @ abstractarray.jl:840 within `similar` @ abstractarray.jl:841
; │││┌ @ boot.jl:468 within `Array` @ boot.jl:459
      %4 = bitcast [4 x {}*]* %gcframe9 to i64*
      store i64 8, i64* %4, align 16
      %5 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 1
      %6 = bitcast {}** %5 to {}***
      %7 = load {}**, {}*** %3, align 8
      store {}** %7, {}*** %6, align 8
      %8 = bitcast {}*** %3 to {}***
      store {}** %gcframe9.sub, {}*** %8, align 8
      %9 = call nonnull {}* inttoptr (i64 1698961392 to {}* ({}*, i64)*)({}* inttoptr (i64 294838384 to {}*), i64 3)
      %10 = bitcast {}* %9 to i64**
      %11 = load i64*, i64** %10, align 8
; │└└└
; │┌ @ array.jl:966 within `setindex!`
    %12 = bitcast i64* %11 to <2 x i64>*
    store <2 x i64> <i64 1, i64 2>, <2 x i64>* %12, align 8
    %13 = getelementptr inbounds i64, i64* %11, i64 2
    store i64 3, i64* %13, align 8
; └└
; ┌ @ abstractarray.jl:2933 within `map`
; │┌ @ array.jl:716 within `collect_similar`
    store {}* %9, {}** %1, align 16
    %14 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 3
    store {}* %9, {}** %14, align 8
    %15 = call nonnull {}* @j__collect_2260({}* nonnull %9, { {}* }* nocapture readonly %2) #0
    %16 = load {}*, {}** %5, align 8
    %17 = bitcast {}*** %3 to {}**
    store {}* %16, {}** %17, align 8
; └└
  ret {}* %15
}

julia> @code_llvm h()
;  @ REPL[19]:1 within `h`
; Function Attrs: uwtable
define nonnull {}* @julia_h_2261() #0 {
pass.2:
  %gcframe9 = alloca [4 x {}*], align 16
  %gcframe9.sub = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 0
  %0 = bitcast [4 x {}*]* %gcframe9 to i8*
  call void @llvm.memset.p0i8.i32(i8* noundef nonnull align 16 dereferenceable(32) %0, i8 0, i32 32, i1 false)
  %1 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 2
  %2 = bitcast {}** %1 to { {}* }*
  %3 = call {}*** inttoptr (i64 1699154720 to {}*** ()*)() #3
; ┌ @ array.jl:126 within `vect`
; │┌ @ array.jl:679 within `_array_for` @ array.jl:676
; ││┌ @ abstractarray.jl:840 within `similar` @ abstractarray.jl:841
; │││┌ @ boot.jl:468 within `Array` @ boot.jl:459
      %4 = bitcast [4 x {}*]* %gcframe9 to i64*
      store i64 8, i64* %4, align 16
      %5 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 1
      %6 = bitcast {}** %5 to {}***
      %7 = load {}**, {}*** %3, align 8
      store {}** %7, {}*** %6, align 8
      %8 = bitcast {}*** %3 to {}***
      store {}** %gcframe9.sub, {}*** %8, align 8
      %9 = call nonnull {}* inttoptr (i64 1698961392 to {}* ({}*, i64)*)({}* inttoptr (i64 294838384 to {}*), i64 3)
      %10 = bitcast {}* %9 to i64**
      %11 = load i64*, i64** %10, align 8
; │└└└
; │┌ @ array.jl:966 within `setindex!`
    %12 = bitcast i64* %11 to <2 x i64>*
    store <2 x i64> <i64 1, i64 2>, <2 x i64>* %12, align 8
    %13 = getelementptr inbounds i64, i64* %11, i64 2
    store i64 3, i64* %13, align 8
; └└
; ┌ @ Untitled-6:4 within `FixLast`
; │┌ @ Untitled-6:4 within `#_#24`
; ││┌ @ abstractarray.jl:2933 within `map`
; │││┌ @ array.jl:716 within `collect_similar`
      store {}* %9, {}** %1, align 16
      %14 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 3
      store {}* %9, {}** %14, align 8
      %15 = call nonnull {}* @j__collect_2263({}* nonnull %9, { {}* }* nocapture readonly %2) #0
      %16 = load {}*, {}** %5, align 8
      %17 = bitcast {}*** %3 to {}**
      store {}* %16, {}** %17, align 8
; └└└└
  ret {}* %15
}

julia> @code_llvm i()
;  @ REPL[20]:1 within `i`
; Function Attrs: uwtable
define nonnull {}* @julia_i_2264() #0 {
pass.2:
  %gcframe9 = alloca [4 x {}*], align 16
  %gcframe9.sub = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 0
  %0 = bitcast [4 x {}*]* %gcframe9 to i8*
  call void @llvm.memset.p0i8.i32(i8* noundef nonnull align 16 dereferenceable(32) %0, i8 0, i32 32, i1 false)
  %1 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 2
  %2 = bitcast {}** %1 to { {}* }*
  %3 = call {}*** inttoptr (i64 1699154720 to {}*** ()*)() #3
; ┌ @ array.jl:126 within `vect`
; │┌ @ array.jl:679 within `_array_for` @ array.jl:676
; ││┌ @ abstractarray.jl:840 within `similar` @ abstractarray.jl:841
; │││┌ @ boot.jl:468 within `Array` @ boot.jl:459
      %4 = bitcast [4 x {}*]* %gcframe9 to i64*
      store i64 8, i64* %4, align 16
      %5 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 1
      %6 = bitcast {}** %5 to {}***
      %7 = load {}**, {}*** %3, align 8
      store {}** %7, {}*** %6, align 8
      %8 = bitcast {}*** %3 to {}***
      store {}** %gcframe9.sub, {}*** %8, align 8
      %9 = call nonnull {}* inttoptr (i64 1698961392 to {}* ({}*, i64)*)({}* inttoptr (i64 294838384 to {}*), i64 3)
      %10 = bitcast {}* %9 to i64**
      %11 = load i64*, i64** %10, align 8
; │└└└
; │┌ @ array.jl:966 within `setindex!`
    %12 = bitcast i64* %11 to <2 x i64>*
    store <2 x i64> <i64 1, i64 2>, <2 x i64>* %12, align 8
    %13 = getelementptr inbounds i64, i64* %11, i64 2
    store i64 3, i64* %13, align 8
; └└
; ┌ @ Untitled-6:60 within `Fix`
; │┌ @ Untitled-6:60 within `#_#7`
; ││┌ @ Untitled-6:60 within `macro expansion`
; │││┌ @ abstractarray.jl:2933 within `map`
; ││││┌ @ array.jl:716 within `collect_similar`
       store {}* %9, {}** %1, align 16
       %14 = getelementptr inbounds [4 x {}*], [4 x {}*]* %gcframe9, i64 0, i64 3
       store {}* %9, {}** %14, align 8
       %15 = call nonnull {}* @j__collect_2266({}* nonnull %9, { {}* }* nocapture readonly %2) #0
       %16 = load {}*, {}** %5, align 8
       %17 = bitcast {}*** %3 to {}**
       store {}* %16, {}** %17, align 8
; └└└└└
  ret {}* %15
}
1 Like

That’s surprising and unusual. Do you know when it might not be done (except for errors/exceptions)? In the docs I find this constructor-line as part of a struct:

OrderedPair(x,y) = x > y ? error("out of order") : new(x,y)

It returns the OrderedPair type, or:

julia> OrderedPair(2, 1)
ERROR: out of order
[..]

julia> typeof(ans)  # Strangely for one such case I got OrderedPair
DataType

I believe in e.g. C++ constructors must construct an object of its class. I believe we might wan to restrict Julia to guarantee the same, i.e. their type (or an exception), and even if not, for autocompletion-purposes, couldn’t we assume it?

Can’t autocomplete be feasible, and useful, if it’s just returning (only) the most likely operations? Even if the list is very long and we show all of it, then if it’s sensibly ordered? I actually care more about that nr. 2, rather than the operators, nr. 1. Is something available in VS Code or elsewhere (i.e. to look up methods, as meant here)?

1 Like

I think this is a good attitude to take toward autocomplete: where a conflict arises between accuracy and usefulness, unlike a compiler, the right choice is often to err toward usefulness. It’s like a Google search; being great most of the time is better than being perfect never.

This is a fine thing for an IDE, it’s not a good principle for a language.

Here’s a fun way to do currying that doesn’t require a macro or changes to the parser. It’s basically the same as using a FixArgs functor, but with nicer syntax.

Here is how it works. To curry a function foo, wrap it in c(), and then call c(foo) they same way that you would call a function using the underscore currying PR, except use .. instead of _.

struct FreeArg end
const .. = FreeArg()

function c(f)
    function(arg_specs...)
        function(free_args...)
            _, args = foldl(arg_specs; init=(free_args, ())) do (free_args, args), arg_spec
                if arg_spec == ..
                    arg, free_args... = free_args
                else
                    arg = arg_spec
                end
                (free_args, (args..., arg))
            end
            f(args...)
        end
    end
end

Here it is in action:

[4, 9, 16] |>
    c(map)(sqrt, ..) |>
    c(filter)(iseven, ..)

It works fairly well for piping, but not so well for small anonymous functions with operators. E.g., compare the following underscore anonymous functions

_.x
_ > 2

to how they would look with the c() function:

c(getproperty)(.., :x)
c(>)(.., 2)

I haven’t done any performance testing. Of course this approach could be modified so that c(foo) returns a FixArgs functor instead of an anonymous function.

If I hijack Base.adjoint, I can make the syntax look almost exactly like the underscore syntax:

[4, 9, 16] |>
    map'(sqrt, ..) |>
    filter'(iseven, ..)
Code for `adjoint` as currying operator
function Base.adjoint(f)
    function(arg_specs...)
        function(free_args...)
            _, args = foldl(arg_specs; init=(free_args, ())) do (free_args, args), arg_spec
                if arg_spec == ..
                    arg, free_args... = free_args
                else
                    arg = arg_spec
                end
                (free_args, (args..., arg))
            end
            f(args...)
        end
    end
end

If you prefer, you can use (typed \square[TAB]) instead of ..:

const □ = FreeArg()

[4, 9, 16] |>
    map'(sqrt, □) |>
    filter'(iseven, □)
6 Likes

I thought about hijacking adjoint too! It looks like a super clean way to indicate partial evaluation.

My concern was that there could be some callable objects for which a mathematical adjoint or transpose has meaning.

Yeah, hijacking adjoint is not something Base would do, but it might be alright in a package. :slight_smile:

1 Like

ooh I like this approach. Explicitly annotating which functions need to be curried seems to be a good way to get around the concerns “when do we stop parsing blocks”

Obviously can’t use ', but maybe there is a suitable unicode character? I somewhat like the look of \wr, and it’s easy to type too

julia> [4, 9, 16] |>
           map≀(sqrt, □) |>
           filter≀(iseven, □)

Or maybe \tricolon

julia> [4, 9, 16] |>
           map⁝(sqrt, □) |>
           filter⁝(iseven, □)
julia> [4, 9, 16] |>
           map🦑(sqrt, □) |>
           filter🦑(iseven, □)
1 Like

Tricolon is not bad! I picked ' because it’s the only unary postfix operator that I know of. (I’m not sure if unicode suffixes work for '…) So to convert my c() function to ', all I had to do was change the name of the function to Base.adjoint. But I think the currying code could be written as a binary function that takes in a function and a tuple of arguments.

Something else I like about how this adjoint hijack works:

f = filter'(isodd, [1,2,3])
f()

:wink:

Oops! :sweat_smile:

No, it’s clean! It’s the way it should be, I like it!

I’m somewhat of a fan of a postfix operator, because (if the operator was friendly with it) it could allow you to operate on the infix operators, something like □ >' 5.

Is it just me, or is (\dagger) always an invalid character, and therefore not taken? :thinking:

\dagger is frequently used to denote Hermitian operators, among probably other things, so better not to touch it as I’m sure some people are using it.

What I mean to say is:

Observe the behavior for special characters, for example, (\prime):

julia> ′
ERROR: syntax: invalid character "′" near column 1

julia> f′
ERROR: UndefVarError: f′ not defined

Now compare with (\dagger):

julia> †
ERROR: syntax: invalid character "†" near column 1

julia> f†
ERROR: syntax: invalid character "†" near column 2

It looks like people can’t use it, so it looks like it’s free for the taking!

BigGrinGIF