[ANN] Metadata.jl

Package Announcements
,
1

Hi everyone,

I’d like to announce the Metadata.jl. It attempts to provide a generic and common interface for attaching and extracting metadata. This is mostly done by assuming that metadata has Symbol keys. Metadata can be attached to anything using attach_metadata and extracted using metadata

julia> using Metadata

julia> x = attach_metadata(ones(2, 2), Dict{Symbol,Any}(:x => 1, :y => 2, :suppress=> [:x]))
2×2 attach_metadata(::Array{Float64,2}, ::Dict{Symbol,Any}
  • metadata:
     y = 2
     x = <suppressed>
)
 1.0  1.0
 1.0  1.0

julia> metadata(x) == Dict{Symbol,Any}(:x => 1, :y => 2, :suppress=>[:x])
true

julia> metadata(x, :x)
1

julia> metadata!(x, :z, 3);

julia> x
2×2 attach_metadata(::Array{Float64,2}, ::Dict{Symbol,Any}
  • metadata:
     y = 2
     z = 3
     x = <suppressed>
)
 1.0  1.0
 1.0  1.0

It also provides macros inspired by the design of Base.Doc and this converstion. These macros use the object id (objectid) of an instance to store metadata in a global dictionary within a given module. In the following example you can see this permits attaching metadata to a type without having to wrap it in a structure.

julia> x = ones(2,2);

julia> @attach_metadata(x, Dict{Symbol,Any}(:x => 1, :y => 2))

julia> @metadata(x)
Dict{Symbol,Any} with 3 entries:
  :y => 2
  :x => 1

julia> @metadata!(x, :z, 3)
Dict{Symbol,Any} with 4 entries:
  :y => 2
  :z => 3
  :x => 1

This isn’t an exhaustive list of all the methods provided (more on that in the README and docs), but hopefully it’s a helpful place for most people to start.

9 Likes
2

Thanks,

can you please provide information about what this package does that goes beyond an implementation of a wrapper type Metadata, which includes two fields like

struct Metadata
parent::X 
meta::M 
end

?

@attach_metadata(x, Dict{Symbol,Any}(:x => 1, :y => 2))

How do yuo handle cases where x itself is a julia struct with field :x? Then what does mx.x do? Does it give you the metadata entry or the subfield entry “x” of the parent x?

1 Like
3

can you please provide information about what this package does that goes beyond an implementation of a wrapper type Metadata , which includes two fields like

It provides a couple wrapper types for things in base taking advantage of the interface, but the purpose of the package isn’t really those wrapper types. Rather, if someone creates a new type there is now a common way for them to approach attaching metadata.

How do yuo handle cases where x itself is a julia struct with field :x ? Then what does mx.x do? Does it give you the metadata entry or the subfield entry “x” of the parent x ?

In an effort to not break things, I have dot operators default to the parent’s fields, but you can still access metadata using metadata.

julia> x = (foo = 1, bar = 2)
(foo = 1, bar = 2)

julia> y = attach_metadata(x, (foo = 3,))
attach_metadata((foo = 1, bar = 2), ::NamedTuple{(:foo,),Tuple{Int64}})
  • metadata:
     foo = 3

julia> y.foo
1

julia> metadata(y, :foo)
3

So the dot operator is probably how you want to access metadata if you’re just accessing things as an everyday user, but if you’re developing code where you want ensure metadata is returned in case the user does something like above then use metadata. It’s pretty much the same situation you get with dot operators for field access and getfield (i.e. you can only guarantee you’re returning a structures field using getfield).

Note that @attach_metadata(x, Dict{Symbol,Any}(:x => 1, :y => 2)) wouldn’t enable access to metadata with dot operators because x isn’t wrapped in a new structure. You would have to use @metadata(x, :x) to access metadata instead. However, you could define a type like this to enable that behavior.

struct NewType
     parent
     parent_module::Module
end

function Base.getproperty(x::NewType, k::Symbol)
    if hasproperty(x.parent, k)
        return getproperty(x.parent, k)
    else
        return Metadata.global_metadata(x, k, x.parent_module)
    end
end
3 Likes
4

Thanks for the reply,

In DrWatson we’ve been discussing about metadata for a long time: https://github.com/JuliaDynamics/DrWatson.jl/issues/151 one of the devs @sebastianpech has even made a package draft for incorporating metadata in simulation environments: https://github.com/sebastianpech/DrWatsonSim.jl

I think it would be useful, both for us, but also for you, if you participate in the discussion!

3 Likes
5

Minor update inspired by this recent Invenia post. Now there is Metadata.test_wrapper so that if you have some unique data type T and and a new dedicated structure W for binding T to metadata you can call Metadata.test_wrapper(W, x) to run an interface test.

1 Like