[ANN] About.jl

About.jl

I’d like to announce a package that I’ve been tinkering on for the past few months. It’s a response to frequently wanting to know more about the structure or nature of objects in Julia, whether that be:

  • How a struct is represented in-memory
  • How large an object is
  • How the bits of a number relate to its value
  • What the compiler knows about a function
  • and more.

The About package exports a single function about, that tells you about any object you pass to it. This makes heavy use of StyledStrings and JuliaSyntaxHighlighting to produce (what I consider to be) pretty outputs designed for human consumption (as opposed to the functions like fieldoffset that aren’t terribly informative at a glance).

The use of StyledStrings also makes the formatting used customisable, in that you can configure elements of the styling through the file ~/.julia/config/faces.toml (see: StyledStrings · Styled Strings).

It’s designed to be easily extendable by packages, which I invite to (when sensible) create package extensions that implement specialised memorylayout and elaboration methods :slight_smile:

There’s a bit more I’d like it to do in due course, but I’m pretty happy with what it does now.

Sample screenshots

Take it for a spin —the module and function(s) all have decent docstrings— and let me know if you’ve got any ideas :grinning:

150 Likes

This package is fantastic! I’m wondering if it’s positioned as a superior replacement and enhancement for Julia REPL’s built-in help?> .

5 Likes

This is great, I’ll be using it all the time! I was briefly a bit confused when trying to compare the outputs of sum([1,2,3]) and sum((1,2,3)). If you want to about the tuple version, it seems you need to write:

about(sum, Tuple{Tuple{Int64, Int64, Int64}})

or you’ll get the (nonexistant) information for sum(1,2,3).

This looks great! Perhaps it would be appropriate for it to implement a hotkey similar to ], ?, and ; for quick access?

2 Likes

Looks great! Thank you for doing this! Just promoted:

Twitter

mastodon

4 Likes

I agree it looks fantastic. Though I think About.jl’s MPL-2.0 license would make it difficult to be adopted like that. Don’t want to derail this thread though, this was discussed recently in a different thread, specifically this post addresses adoption:

Not really, help?> is all about docstrings, and this is more about introspection.

2 Likes

This is a little tricky, this seems annoying but I wrote this so it could also be used with the same signature as functions like methods and return_types in Base, as well as the “more convenient” form I demonstrated above.

Hmmm, I’m not sure what would be best to do here.

Could you do

@about sum((1, 2, 3))

?

1 Like

Possibly, I’ve been thinking of adding an @about macro for fun, but the main roadblock is resolving whether an @about <func call> invocation should give information on the function called, or the result.

Thinking of cases like @about Float16(1.234) makes this harder to me.

My first instinct is that the above should report on the function called, and

about(Float16(1.234))

should handle the result.

8 Likes

Really loving this, thank you very much!

Have just filed a couple of issues… :smile_cat:

To everyone filing issues and PRs, they’re much appreciated :heart_eyes:.

It’s great to see people taking me up on the request to help me make About.jl even more useful (and less buggy :stuck_out_tongue_closed_eyes:).

I might take a bit to get around to them though, I’m in a rather busy spell currently.

2 Likes

Very nice idea and UI!

Unfortunately this seems to fail when it is most needed, i.e. when datatypes have nontrivial (non C-compatible) layout.

This is especially unfortunate since, to my knowledge, the beyond-C layout rules in julialang are not documented, and there is no built-in introspection facility. Can someone from the core team correct me if I’m wrong on that?

Two examples on 1.10.4:

julia> using About
julia> mutable struct UnionTags
       x::Union{Int,UInt}
       end
julia> about(UnionTags(1))
UnionTags (mutable) (<: Any), occupies 16B directly (referencing 24B in total)
 x::Union{Int64, UInt6… 16B Ptr? 1

 ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
                                   *                                   

 * = Pointer (8B)
julia> mutable struct AtomicHiddenLock
       @atomic x::Tuple{Int,Int,Int}
       end
julia> about(AtomicHiddenLock((1,2,3)))
AtomicHiddenLock (mutable) (<: Any), occupies 40B.
 x::Tuple{Int64, Int64… 24B «struct» (1, 2, 3)

 ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
                    24B      

The desired behavior should be to exhibit the true memory layout that is otherwise hard to figure out. This includes:

  • For every piece of information, we want to know where it is stored (“data of field X is here; a union-tag for X is there; a hidden implicit lock protecting protecting field xyz there”)
  • For every byte in the memory layout, we want to know where it belongs to (e.g. “structure padding”, “hidden implicit lock for field xyz”, “small-union-tag”)
  • If we can’t figure that out, it would be nice to have sanity checks. Then one can at least print a warning that the layout is likely imprecise.

(ceterum censeo, hidden locks for atomics suck – if it’s too large for lock cmpxchg16b then it should be boxed!)

2 Likes

This is a consequence of what I get out of the introspection tools available

# Julia 1.10.4
julia> mutable struct UnionTags
       x::Union{Int,UInt}
       end

julia> u = UnionTags(1)
UnionTags(1)

julia> Base.summarysize(u)
24

julia> sizeof(u)
16

That would be great! If somebody would be willing to help implement this, or just work out how this can be determined within About I’d be thrilled.

1 Like

This looks really good. It would be nice if some api existed to extract information instead of having it printed to the standard output.

At the moment I use this function to collect the number of bytes used by a reasonably complex data structure:

function allbytes(a)
    T = typeof(a)
    return if isbitstype(T)
        sizeof(a)
    else
        return if fieldcount(T) == 0
            if length(a) > 0
                sum(allbytes(a[i]) for i in eachindex(a))
            else
                sizeof(a)
            end
        else
            sum(allbytes(getfield(a, fieldname(T, i))) for i in 1:fieldcount(T))
        end
    end
end

Compared to About, allbytes simplifies by not considering the size of the record that represents the structure, only the size of the fields (arrays and vectors and such) , which is where most of the storage really is.

Also, I couldn’t get About to work with a sparse matrix. Or with a simpler case:

julia> struct A
       v::Vector{Float64}
       end

julia> A([1.0, 2.0])
A([1.00000e+00, 2.00000e+00])

julia> a = A([1.0, 2.0])
A([1.00000e+00, 2.00000e+00])

julia> about(a)
A (<: Any), occupies 8B directly (referencing 64B in total)
T = A
(fieldname(T, i), fieldtype(T, i), hassizeof(fieldtype(T, i))) = (:v, Vector{Float64}, true)
ERROR: Type Array does not have a definite size.
Stacktrace:
  [1] sizeof(x::Type)
1 Like

Version 1.0.1 is out (imminently), with 14 bugfixes and the first contribution :grinning:

Slipped into the bugfix release is one technically non-bug change, we now show a preview of the memory layout of Arrays on 1.11+. Previously you had to explicitly inspect the underlying Memory.

12 Likes