I think this is interesting (and I do think Observables could do with nicer syntax!) though this seems like a special enough case that Observables should just define its own special purpose macro.
For Observables you need to distinguish between variables which are Observable vs those which are not (for the purposes of creating the closure). This is a kind of “escaping” so it might be natural to use $x syntax. For example:
x = 1
a, b = Observable(1), Observable(2)
c = @Observable (a+b)*a + $x
Alternatively, if you think most variables will be captured (rather than form arguments to the closure) you could just reuse the existing syntax for observable getindex, but automatically lift it out into a closure:
c = @Observable (a[] + b[])*a[] + x
A third option would be to just lift the names of the things being observed out of a closure argument list to also take them from the current scope. This is general and the syntax is very evocative:
c = @Observable (a,b) -> (a + b)*a + x
You could even allow a combination of these forms, for example the first for short one-liners, and the third for more general cases
Oh right, do syntax for arbitrary slots; I hadn’t considered that, that’s a really cool idea!
Now that you’ve mentioned it, I really want to do something like this though I fear we can’t use _ because it changes the meaning of _ in a weird way. It also disallows using _ for it’s “normal” meaning which could be really handy for higher order functions which accept multiple closures.
So I think we need a different marker. An ugly but straightforward possibility is just to use _do:
It’s interesting that there’s several precedents here which are quite different. In yet another case, Elixir uses &, Kernel.SpecialForms — Elixir v1.15.7, to get things like &(&1 + &2). Amusingly / surprisingly this syntax is available in Julia (though it may eventually be used for Ref?). In some ways I find &1 more visually pleasing than _1 but I do think the “outer” placement of the @_ is quite nice.
Mathematica uses # or #1 for our _ and _1, but it’s arguably a bit of a different case if the vast majority of Mathematica users use it via the standard UI where # is highlighted and in a different font Functional Operations—Wolfram Language Documentation
I was thinking “two step” lowering which (conceptually) transforms above pieces of code to
@_ ans = mapreduce(f, _, xs)
ans() do x, y
x + y
end
@_ ans = reduce(op, Filter(_), xs)
ans() do x
x > 0
end
and then to
ans = _1 -> mapreduce(f, _1, xs)
ans() do x, y
x + y
end
ans = _1 -> reduce(op, Filter(_1), xs)
ans() do x
x > 0
end
The last step already works so I guess the fishy part is the first one? That is to say, inserting ans = alters the boundary of the implicit anonymous function?
Yes that’s a good way to state it! The way Expr(:do) is parsed this is also very easy and natural to implement with the Expr(:call) nesting inside the Expr(:do) in the AST.
Is there anything else this could mean?
Thinking about it, there’s a choice, but the other alternative is relatively useless:
@_ map(__, xs) do x
x+y
end
Could be the current meaning:
__1 -> (map(__1, xs) do x
x+y
end)
but this is only really helpful inside piping or composition, IMO and one wouldn’t use a do block in conjunction with that. (__ can also be used to create an anonymous function but that’s not the core use case for @_ as I see it.)
Compared this to the more useful meaning:
f = (__1 -> map(__1, xs))
f() do x
x+y
end
This second one seems fine, though there’s one oddity which is the need to insert an extra Expr(:call) (the () in f()). But I think that’s ok. Also __2 can’t mean something sensible in this case, but that’s probably not a big deal, I’ll just make it an error.
I think in practice I’ll lower this slightly differently to avoid creating another nested closure:
let __1 = (x,)->begin # The closure from `Expr(:do)`
x+y
end
map(__1, xs)
end
I know you’ve thought a lot about mapreduce and related functions. Do you have a compelling example of how this would be used somewhere which I could put in the docs?
Yes. The rule is that the boundary for _ is inside the “outer” call, where piping (and do in the PR) doesn’t count for the purposes of determining “outerness”
Actually I think not being able to use do with |> is OK I was just wondering what the rule is. I was probably mixing it with some other proposals.
So how do you apply the rules to lower @_ map("X $(repeat(_,_))", ["a","b","c"], [1,2,3]) in the documentation to map((a, b) -> "X $(repeat(a, b))", ["a","b","c"], [1,2,3]) and not map("X $((a, b) -> repeat(a, b))", ["a","b","c"], [1,2,3]) or map("X $(repeat(identity, identity))", ["a","b","c"], [1,2,3])?
This is no longer a valid example :-/ Read the dev docs rather than stable docs — people further up the thread have (mostly) persuaded me that _ always means _1 and that’s what’s on master.
Hmm. The need to add an extra Expr(:call) implicitly in the more specialized lowering of do does make me worry that the most convenient interpretation of __ may not be entirely consistent.
After thinking about it for longer, I actually am disagreeing with myself now. In particular, if _ indicates a different argument upon each use, then one might expect __ to have this same property; it seems clear you wouldn’t want that to happen to __. On that front does it make sense to have __1, __2, etc…???
Yes, this was actually implemented from the start If you look at the implementation you’ll see it’s the exact same code which lowers _ and __ (and associated numbered versions) — it’s just some regexes which differ and the part of the expression that they’re applied to. So it’s all quite uniform and things like @_ __₁ ^ __₁ work.
@c42f Thanks, this is a cool package! I just commented on the lengthy discussion of PR #24990, so I don’t want to duplicate everything I said there. But basically, I like this approach, except for the double underscores. I feel like the double underscore syntax adds complexity to the syntax in order to solve a problem not directly related to syntax for closures. Namely, the base pipe simply isn’t powerful enough. Syntax like data |> d -> map(x -> x.f, d) demonstrates that we need better pipes.
It would be great if we had
x |> f(a, b) # translated to f(x, a, b)
and
x |>> f(a, b) # translated to f(a, b, x)
Then your example would be written as
@_ t |>>
filter(_.age > 27) |>>
group(_.sex) |>>
map(length)
Would it be possible to add a new macro (or update the @_ macro) to support the more advanced pipes, and thus support the syntax shown in the pipeline above?
(I’m guessing that a symbol other than |>> would have to be found, since |>> doesn’t parse as a binary operator.)
. But can’t it be done with single 
