Fields with abstract types in Base and StdLib

Internals & Design
1

Hi!

eachline was slow because the EachLine stream field has the abstract type IO and it can be solved by using a type parameter for stream in EachLine. So, I looked for other structures in Base and StdLib (Julia 0.7.0-beta2.81) with abstract type fields that could lead to hidden performance issues. Here are the bash lines I’ve used to find them and the list of the files that define structs with abstract type fields (with small editions to make them more leggible or complete):

~/bin/julia-fdad5051ff/share/julia/base$ ag "^abstract type .* end" | cut -d" " -f3 | cut -d"{" -f1 > ~/abstract_types.txt 
~/bin/julia-fdad5051ff/share/julia/base$ for t in $(cat ~/abstract_types.txt); do ag -B 3 "^\s+[a-zA-Z0-9]+::$t\s*$"; done

logging.jl
359-struct LogState
360-    min_enabled_level::LogLevel
361:    logger::AbstractLogger

strings/string.jl
8-struct StringIndexError <: Exception
9-    string::AbstractString
10:    index::Integer

logging.jl
500-struct SimpleLogger <: AbstractLogger
501:    stream::IO

multimedia.jl
201-struct TextDisplay <: AbstractDisplay
202:    io::IO

process.jl
299-mutable struct Process <: AbstractPipe
300-    cmd::Cmd
301-    handle::Ptr{Cvoid}
302:    in::IO
303:    out::IO
304:    err::IO

io.jl
845-struct EachLine
846:    stream::IO

process.jl
74-struct OrCmds <: AbstractCmd
75:    a::AbstractCmd
76:    b::AbstractCmd
80-struct ErrOrCmds <: AbstractCmd
81:    a::AbstractCmd
82:    b::AbstractCmd
86-struct AndCmds <: AbstractCmd
87:    a::AbstractCmd
88:    b::AbstractCmd
157-struct CmdRedirect <: AbstractCmd
158:    cmd::AbstractCmd

~/bin/julia-fdad5051ff/share/julia/base$ cd ../stdlib/v0.7/
~/bin/julia-fdad5051ff/share/julia/stdlib/v0.7$ ag "^abstract type .* end" | cut -d" " -f3 | cut -d"{" -f1 >> ~/abstract_types.txt 
~/bin/julia-fdad5051ff/share/julia/stdlib/v0.7$ for t in $(cat ~/abstract_types.txt); do ag -B 3 "^\s+[a-zA-Z0-9]+::$t\s*$"; done

LinearAlgebra/src/cholesky.jl
46-struct CholeskyPivoted{T,S<:AbstractMatrix} <: Factorization{T}
47-    factors::S
48-    uplo::Char
49-    piv::Vector{BlasInt}
50-    rank::BlasInt
51:    tol::Real

Distributed/test/topology.jl
37-mutable struct TopoTestManager <: ClusterManager
38:    np::Integer

Distributed/src/managers.jl
289-struct LocalManager <: ClusterManager
290:    np::Integer

Logging/src/ConsoleLogger.jl
25-struct ConsoleLogger <: AbstractLogger
26:    stream::IO

REPL/src/REPL.jl
1038-mutable struct StreamREPL <: AbstractREPL
1039:    stream::IO

Base64/src/encode.jl
34-struct Base64EncodePipe <: IO
35:    io::IO

Base64/src/decode.jl
35-struct Base64DecodePipe <: IO
36:    io::IO

Distributed/src/process_messages.jl
5-mutable struct RemoteValue
6:    c::AbstractChannel

REPL/src/REPLCompletions.jl
54-struct DictCompletion <: Completion
55:    dict::AbstractDict

LinearAlgebra/src/cholesky.jl
46-struct CholeskyPivoted{T,S<:AbstractMatrix} <: Factorization{T}
47-    factors::S
48-    uplo::Char
49-    piv::Vector{BlasInt}
50-    rank::BlasInt
51:    tol::Real

Distributed/src/cluster.jl
53-mutable struct Worker
54-    id::Int
55-    del_msgs::Array{Any,1}
56-    add_msgs::Array{Any,1}
57-    gcflag::Bool
58-    state::WorkerState
59-    c_state::Condition      # wait for state changes
60-    ct_time::Float64        # creation time
61-    conn_func::Any          # used to setup connections lazily
62-
63-    r_stream::IO
64-    w_stream::IO
65-    w_serializer::ClusterSerializer  # writes can happen from any task hence store the
66-                                     # serializer as part of the Worker object
67:    manager::ClusterManager

REPL/src/LineEdit.jl
69-mutable struct PromptState <: ModeState
70:    terminal::AbstractTerminal
1542-mutable struct SearchState <: ModeState
1543:    terminal::AbstractTerminal
1599-mutable struct PrefixSearchState <: ModeState
1600:    terminal::AbstractTerminal

REPL/src/REPL.jl
199-mutable struct BasicREPL <: AbstractREPL
200:    terminal::TextTerminal
304-mutable struct LineEditREPL <: AbstractREPL
305:    t::TextTerminal

Maybe those structs can use type parameters or use more specific types for their fields.

Cheers,

5 Likes