Question about any possible text manipulation tools for 'fine-tuning' introspection (reflection) output

Is it possible to use some kind of ‘pattern/string match’ in the REPL when inspecting dispatch tables of functions with hundreds of methods.

Like…

methods(convert) | grep 'some pattern like <: Number or :> Number'

I guess text search is not ideal, but could be useful for searching on which file defined them or the type arguments.

I guess I am basically asking for a way to search for a subset of the methods (off course I can directly use the @which macro to inspect any specific dispatch).

I guess I am basically asking, if I can combine the julia REPL with the unix shell “simultaneosly” somehow, or (more realistically :smiley:) if Julia has some text manipulation tools built-in that can be used to “manipulate” text output (like grep) ? I all ready found pwd(), cd(“some-rel-path”) that mimick the shell filesystem tools.

for m in methods(convert)
       if occursin("Float",string(m.sig)); println(m); end
end
1 Like

Here are some pipe-oriented options:

  1. methods(convert) returns an array. You can use the full power of the language to filter the elements:

    julia> filter(contains("Number")∘string, methods(convert))
    [1] convert(::Type{T}, x::T) where T<:Number in Base at number.jl:6
    [2] convert(::Type{T}, x::Number) where T<:Number in Base at number.jl:7
    [3] convert(::Type{T}, x::Base.TwicePrecision) where T<:Number in Base at twiceprecision.jl:262
    [4] convert(::Type{T}, x::AbstractChar) where T<:Number in Base at char.jl:185
    [5] convert(::Type{T}, index::CartesianIndex{1}) where T<:Number in Base.IteratorsMD at multidimensional.jl:136
    [6] convert(::Type{AbstractChar}, x::Number) in Base at char.jl:183
    [7] convert(::Type{T}, x::Number) where T<:AbstractChar in Base at char.jl:184
    [8] convert(::Type{Base.TwicePrecision{T}}, x::Number) where T in Base at twiceprecision.jl:263
    

    Now the same thing with the pipe operator |> :

    methods(convert) |> x->filter(contains("Number")∘string, x)
    

    The nice thing with this method is that you still have Method objects as a result, not just a text representation.

  2. Convert the whole output to text and still use the language on the result:

    julia> methods(convert) |> string |> (x->split(x, '\n')) |> (x->filter(contains("Number"), x))
    8-element Vector{SubString{String}}:
     "[15] convert(::Type{T}, x::T) where T<:Number in Base at number.jl:6"
     "[16] convert(::Type{T}, x::Number) where T<:Number in Base at number.jl:7"
     "[17] convert(::Type{T}, x::Base" ⋯ 34 bytes ⋯ "n Base at twiceprecision.jl:262"
     "[18] convert(::Type{T}, x::Abst" ⋯ 17 bytes ⋯ "<:Number in Base at char.jl:185"
     "[19] convert(::Type{T}, index::" ⋯ 50 bytes ⋯ "rsMD at multidimensional.jl:136"
     "[44] convert(::Type{AbstractChar}, x::Number) in Base at char.jl:183"
     "[45] convert(::Type{T}, x::Numb" ⋯ 17 bytes ⋯ "ractChar in Base at char.jl:184"
     "[52] convert(::Type{Base.TwiceP" ⋯ 34 bytes ⋯ "n Base at twiceprecision.jl:263"
    

    (Here I split the text into lines but you don’t have to.)

  3. Actually use the system’s grep:

    julia> methods(convert) |> string |> IOBuffer |> x->pipeline(x, `grep Number`) |> run
    [15] convert(::Type{T}, x::T) where T<:Number in Base at number.jl:6
    [16] convert(::Type{T}, x::Number) where T<:Number in Base at number.jl:7
    [17] convert(::Type{T}, x::Base.TwicePrecision) where T<:Number in Base at twiceprecision.jl:262
    [18] convert(::Type{T}, x::AbstractChar) where T<:Number in Base at char.jl:185
    [19] convert(::Type{T}, index::CartesianIndex{1}) where T<:Number in Base.IteratorsMD at multidimensional.jl:136
    [44] convert(::Type{AbstractChar}, x::Number) in Base at char.jl:183
    [45] convert(::Type{T}, x::Number) where T<:AbstractChar in Base at char.jl:184
    [52] convert(::Type{Base.TwicePrecision{T}}, x::Number) where T in Base at twiceprecision.jl:263
    Process(`grep Number`, ProcessExited(0))
    
  4. Use TerminalPager.jl. That’s probably the most pragmatic for your use case! With this package you can simply do

    methods(convert) |> pager
    

    and search in the output as with the less command (so pressing / then some text or regular expression will highlight all the matches, and you can jump to the next or previous with n and N).

Also note that in Julia 1.8 you will be able to TAB-complete method arguments in the help mode, for example typing ?convert(Number, and pressing TAB will show all the convert methods that accept a Number as first parameter:

help?> convert(Number,
convert(::Type{T}, x::T) where T<:Number in Base at number.jl:6
convert(::Type{T}, x::Number) where T<:Number in Base at number.jl:7
convert(::Type{T}, x::Base.TwicePrecision) where T<:Number in Base at twiceprecision.jl:273
convert(::Type{T}, x::AbstractChar) where T<:Number in Base at char.jl:185
convert(::Type{T}, index::CartesianIndex{1}) where T<:Number in Base.IteratorsMD at multidimensional.jl:130
convert(::Type{T}, x::T) where T in Base at essentials.jl:215

Finally, if you think the pipe examples above are a bit clumsy you are not alone, hopefully this PR will be merged one day do make it nicer. With that PR the above examples would become:

methods(convert) |> filter(contains("Number")∘string, _)

methods(convert) |> string |> split(_, '\n') |> filter(contains("Number"),  _)

methods(convert) |> string |> IOBuffer |> pipeline(_, `grep Number`) |> run
1 Like

Might also be useful

1 Like

Dead simple. Thx

Elegant :+1:

So many reasons to ‘opt my game’ in how to combine and transform text from within julia, and even subprocess to non-julia (unix) tools. Great answers. Thanks a lot. This is learning Julia by learning Julias library API’s (meta learning :grinning:)