Fixing the Piping/Chaining Issue

Internals & Design
, , , ,
182

Okay I guess I misunderstood what you meant by “actual real-life example.”

The best examples will not be of methods or types from Base, but from packages that create complicated objects with many tightly-specialized methods, for which method discovery will be most appreciated.

I’ll use DataFrames.jl as an example. Keep in mind, this is just a simple example of how autocomplete could behave, solely by acting on types.

Example Possible Autocomplete Behavior

Here's a walked-through example of how an autocomplete *could* work with underscore syntax. (click to expand)

Let’s create an object df = DataFrame(a=1, b=2). When I type

df |>

I should see a (very very long) list of methods appear: those which specialize on typeof(df), followed by methods which specialize on supertype(typeof(df)), followed by supertype(supertype(typeof(df))), etc., sorted by degree of specialization. The list is about two thousand entries long, something like this:

df |>
  append!(df1::DataFrame, df2::AbstractDataFrame; cols, promote)
  append!(df::DataFrame, table; cols, promote)
  copy(df::DataFrame; copycols)

  ⋮ other methods of `DataFrame`
  ⋮ (vdots here to shorten my explanation)

  Array(df::AbstractDataFrame)
  ==(df1::AbstractDataFrame, df2::AbstractDataFrame)
  (Matrix)(df::AbstractDataFrame)

  ⋮ other methods of `AbstractDataFrame`

  ArgumentError(msg)
  AssertionError(msg)
  BoundsError(a)

  ⋮ other methods of `Any`

The fact that we have underscore syntax in the language means I can call any of these methods conveniently using the pipe operator. The list was simply created by calling methodswith of the type and its supertypes, with no attention paid to argument position.

Pressing CTRL+B (or something, some hotkey combination) might change settings. For example, maybe I want to see only methods that act on abstract types, in which case pressing CTRL+B could bring up:

df |>
  Array(df::AbstractDataFrame)
  ==(df1::AbstractDataFrame, df2::AbstractDataFrame)
  (Matrix)(df::AbstractDataFrame)

  ⋮ other methods of `AbstractDataFrame`

  ArgumentError(msg)
  AssertionError(msg)
  BoundsError(a)

  ⋮ other methods of `Any`

But for now, I decide I want to see methods specialized to strictly this concrete type. So I press CTRL+B again and I see:

df |>
  append!(df1::DataFrame, df2::AbstractDataFrame; cols, promote)
  append!(df::DataFrame, table; cols, promote)
  copy(df::DataFrame; copycols)
  delete!(df::DataFrame, inds)
  deleteat!(df::DataFrame, inds::InvertedIndex)
  deleteat!(df::DataFrame, inds::AbstractVector{Bool})

  ⋮

And then I can scroll down the list to find what I’m looking for. One neuron fires in my brain and I remember that the first character is a p. So I type p and I see:

df |> p
  pop!(df::DataFrame)
  popat!(df::DataFrame, i::Integer)
  popfirst!(df::DataFrame)
  prepend!(df1::DataFrame, df2::AbstractDataFrame; cols, promote)
  prepend!(df::DataFrame, table; cols, promote)
  push!(df::DataFrame, row::Union{AbstractDict, NamedTuple}; cols, promote)
  push!(df::DataFrame, row::DataFrameRow; cols, promote)
  push!(df::DataFrame, row; promote)
  pushfirst!(df::DataFrame, row::Union{AbstractDict, NamedTuple}; cols, promote)
  pushfirst!(df::DataFrame, row::DataFrameRow; cols, promote)
  pushfirst!(df::DataFrame, row; promote)

The list is now sufficiently short that I can see the whole thing and remind myself that the function I wanted to call was pushfirst!, and I type u. Now I see:

df |> pu
  push!(df::DataFrame, row::Union{AbstractDict, NamedTuple}; cols, promote)
  push!(df::DataFrame, row::DataFrameRow; cols, promote)
  push!(df::DataFrame, row; promote)
  pushfirst!(df::DataFrame, row::Union{AbstractDict, NamedTuple}; cols, promote)
  pushfirst!(df::DataFrame, row::DataFrameRow; cols, promote)
  pushfirst!(df::DataFrame, row; promote)

I hit <tab> and it autocompletes to push:

df |> push
  push!(df::DataFrame, row::Union{AbstractDict, NamedTuple}; cols, promote)
  push!(df::DataFrame, row::DataFrameRow; cols, promote)
  push!(df::DataFrame, row; promote)
  pushfirst!(df::DataFrame, row::Union{AbstractDict, NamedTuple}; cols, promote)
  pushfirst!(df::DataFrame, row::DataFrameRow; cols, promote)
  pushfirst!(df::DataFrame, row; promote)

Now I type f:

df |> pushf
  pushfirst!(df::DataFrame, row::Union{AbstractDict, NamedTuple}; cols, promote)
  pushfirst!(df::DataFrame, row::DataFrameRow; cols, promote)
  pushfirst!(df::DataFrame, row; promote)

I press <tab> again and the name fully fills out, including the unfixed argument and placing the cursor after the comma. Now I see:

df |> pushfirst!(_, )
  pushfirst!(df::DataFrame, row::Union{AbstractDict, NamedTuple}; cols, promote)
  pushfirst!(df::DataFrame, row::DataFrameRow; cols, promote)
  pushfirst!(df::DataFrame, row; promote)

and the list all but disappears as I fill out the rest of the arguments.

df |> pushfirst!(_, [1, 2])
  pushfirst!(df::DataFrame, row; promote)

The autocomplete has assisted me in finding the method I was looking for, enabling me to search for methods which specialize on its concrete type.

Where m is one of the methods returned by calling methodswith, this simple example uses only m.sig and doesn’t sort by argument position at all. However, it could be imagined to do so.

In addition, it could be imagined to use m.module to sort methods by what module defined them, showing first the methods defined in the same module as this object (using parentmodule(MyType)); m.file to find only the methods which were defined in the same file as this object; or any of the other properties of a Method to return better search results. (and the autocomplete could have a suite of hotkeys, or a settings panel, or some settings popup dialog, to determine how it searches.) It could also use statistical inference based on function call data from GitHub, or even your personal use data, to return better search results.

I hope I don’t have to write my own autocomplete, and I’m totally incompetent, but if I’m pushed hard enough…

1 Like