General partial application in Base?

Internals & Design
6

Necropost!! :wink:

That’s an interesting application, I never thought of it. I use the NamedTuple constructor for similar reasons, to programmatically manipulate field names, types, and values.

These are some additional benefits I see to using a partial application type over constructing on-the-fly lambdas:

Benefits:

  1. Every place in your code that you write a lambda, it causes a new function to be compiled. For example, map(Fix2(^, 2), arr) is preferable to map(x->x^2, arr) because the Fix2{typeof(^), Int64} type has undoubtedly been used elsewhere in the codebase and has been compiled already. It’s wasteful to compile lots of code that does exactly the same thing over and over and over and over and over.
  2. Frequently, this use of a lambda to satisfy partial application will cause a variable to be captured from its environment. (Take map(k->o[k], ks) for example; o is a capture.) As discussed here, if you’re not careful to avoid reassigning the identifier of the captured variable at its parent scope, this can result in substantial performance loss and potentially type instability. The act of passing the capture as an argument to a partial application constructor, e.g. map(Fix1(getindex, o), ks), provides a function barrier that prevents such behavior.
  3. A type dedicated to partial application allows type-specialization. An obvious use is for Base.show, but InverseFunctions.jl provides an interesting use case too (here’s an example). I can also imagine a world in which autocomplete would automatically show the argument names and types as they have been passed through the partial applicator.

Then, if PR#24990 is accepted to use _ underscores for partial application, all of these problems would be solved in a very ergonomic way. The example from #1 could be written map(_^2, arr), and the example from #2 would be map(o[_], ks).

Very, very nice.

I’ve also made a generalized partial application type which the syntax could lower to, which I describe here. In Julia 1.9, we will be getting a currying filter(f::Function) function (which uses Base.Fix1), but no comparable map(f::Function) (which would need a more general partial applicator, which my code satisfies with a FixFirst type).

What has prevented PR#24990 from being accepted has been a desire to get more than just one function call out of the syntax—to define a lambda instead of a partial applicator, so that e.g. you could write filter(_%3==0, arr). At the parser level, this is generally impossible to get right, and moreover, yet another way to write a lambda isn’t all that interesting.

However, as suggested here, a function composition fallback could satisfy this desire. I’ve also explored this idea, and it works very well. (although I don’t have the option to change the fallback behavior so I used dispatch.) Of course, such a fallback would require a generalized partial applicator type.

As I’ve stated at the bottom of this comment, I believe having a general partial applicator type and PR#24990 as syntax sugar to construct it would be part of a more perfect Julia.

1 Like