# \[ANN\] AbstractMetaArrays.jl – A Framework for Metadata-Aware Arrays
**URL:** https://discourse.julialang.org/t/ann-abstractmetaarrays-jl-a-framework-for-metadata-aware-arrays/128123
**Category:** Package Announcements
**Created:** [April 16, 2025, 1:05pm UTC](https://discourse.julialang.org/t/ann-abstractmetaarrays-jl-a-framework-for-metadata-aware-arrays/128123 "2025-04-16T13:05:59Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@marco\_menarini](https://discourse.julialang.org/u/marco_menarini)
#### Post date: [April 16, 2025, 1:05pm UTC](https://discourse.julialang.org/t/ann-abstractmetaarrays-jl-a-framework-for-metadata-aware-arrays/128123/1 "2025-04-16T13:05:59Z")
I’m pleased to announce the registration of [`AbstractMetaArrays.jl`](https://github.com/uriele/AbstractMetaArrays.jl), a Julia package that provides a flexible framework for creating arrays enriched with metadata. The package should become available in the General registry after the standard waiting period.
### Overview
`AbstractMetaArrays.jl` introduces a new abstract type that extends Julia’s array capabilities by allowing arrays to carry additional metadata (`metadata`) and, optionally, column-specific metadata (`colmetadata`). This is particularly useful in scientific computing domains like micromagnetics and photonics, where contextual information (e.g., units, coordinate systems, physical properties, normalization factors) is essential.
The original motivation for this package was the need to:
- **Maintain Dimensional Context** : Perform computations on normalized data while retaining information about original units and normalization factors within the same array structure, facilitating accurate visualization and comparison of results.
- **Handle Diverse Coordinate Systems** : Manage and compare datasets using different conventions (e.g., geocentric vs. geodetic coordinates) without the overhead of continuous conversions, by embedding coordinate system information directly into the array metadata.
but I couldn’t find an existing package that could easily do that out of the box.
### Key Features
- **Metadata Integration** : Attach arbitrary metadata to arrays without compromising performance.
- **Column Metadata Support** : Explicit support for `colmetadata` in `StructArrays` and `StaticArrays` (`SVector{S}` and `MVector{S}`).
- **DataAPI.jl Compatibility** : Extends the existing metadata functions from [`DataAPI.jl`](https://github.com/JuliaData/DataAPI.jl), ensuring seamless integration with Julia’s data ecosystem.
- **Customizable Behavior** : Define how metadata propagates through array operations using traits:
- `ColMetadataTrait`: Indicates if the implementation supports `colmetadata`, independent of the underlying data type.
- `MetadataStyle` and `ColMetadataStyle`: Control the reading and writing access to `metadata` and `colmetadata`.
### Utilities
The package also provides:
- **`SimpleMetaArray`** : A straightforward implementation of a concrete type for quick adoption.
- **`create_metaarray`** : A utility function to easily generate new concrete metadata arrays tailored to specific needs in a format compatible with `DataAPI.jl` specifics.
Additional examples of concrete implementations with different read/write access on metadata are provided in [`test/datastructuretest.jl`](https://github.com/uriele/AbstractMetaArrays.jl/blob/master/test/datastructuretest.jl). These are primarily used for testing metadata access using the trait system but could be helpful for anyone planning to create their own concrete implementation.
### Contributions and Feedback
Contributions are welcome! If you encounter any issues or have suggestions for improvements, feel free to open an issue or submit a pull request on [GitHub](https://github.com/uriele/AbstractMetaArrays.jl).
I’m looking forward to seeing how the community utilizes `AbstractMetaArrays.jl` in their projects.
---
### Author:  [@aplavin](https://discourse.julialang.org/u/aplavin)
#### Post date: [April 16, 2025, 2:31pm UTC](https://discourse.julialang.org/t/ann-abstractmetaarrays-jl-a-framework-for-metadata-aware-arrays/128123/2 "2025-04-16T14:31:15Z")
Arrays with generic metadata are useful indeed, but would be nice to have a comparison with MetadataArrays.jl – so that users could understand the context and any potential differences.
---
### Author:  [@marco\_menarini](https://discourse.julialang.org/u/marco_menarini)
#### Post date: [April 16, 2025, 2:58pm UTC](https://discourse.julialang.org/t/ann-abstractmetaarrays-jl-a-framework-for-metadata-aware-arrays/128123/3 "2025-04-16T14:58:35Z")
The main difference is that it provides an abstract type that forwards all the functions of array, also it allows for different access to read/write access using traits.
The other thing is that `MetadataArrays.jl` does not support colmetadata arrays.
For example, if I just want to know if one particular element of my structure has certain information, I can just query the correct key
This is the usercase for the project I am working on
```julia
# definition of concrete structure
struct GeoArray{T,N,A<:StructArray} <: AbstractMetaArray{T,N,A}
_data::A
_metadata:: ConcreteMetaType
_colmetadata:: Dict{Symbol,ConcreteMetaType}
function GeoArray{T,N}(data::StructArray{T,N}, _metadata::ConcreteMetaType, _colmetadata::Union{Tuple{ConcreteMetaType},ConcreteMetaType}) where {T,N}
# create metaarray takes care of creating the correct format for DataAPI.jl
# where the metadata has the form Dict{String,(Any,Symbol)}, the symbol is necessary
# and if it is not provided is gives as :default, also any key is converted
# to a string as required by the interface.
metainfo=AbstractMetaArrays.create_metaarray(GeoArray{T,N}, data, _metadata, _colmetadata)
return new{T,N,StructArray{T,N}}(data, metainfo...)
end
function GeoArray{T,N}(data::A, _metadata::ConcreteMetaType, _colmetadata::Union{Tuple{ConcreteMetaType},ConcreteMetaType}) where {T,N,A<:AbstractArray{T,N}}
metainfo=AbstractMetaArrays.create_metaarray(GeoArray{T,N}, data, _metadata, _colmetadata)
return new{T,N,StructArray{T,N}}(StructArray(data), metainfo...)
end
end
# tells me that the method support column metatada
AbstractMetaArrays.ColMetadataTrait(::Type{<:GeoArray}) = AbstractMetaArrays.HasColMetadata()
# readonly colmetadata, so that it can only be modified internally by the code
AbstractMetaArrays.ColMetadataStyle(::Type{<:GeoArray}) = AbstractMetaArrays.ReadOnlyColMetadata()
# readonly metadata
AbstractMetaArrays.MetadataStyle(::Type{<:GeoArray}) = AbstractMetaArrays.ReadOnlyMetadata()
const MAJORAXISWGS84 = majoraxis(ellipsoid(WGS84Latest)) |>
x-> uconvert(km, x) |> ustrip
const ECCENTRICITY2WGS84 = eccentricity²(ellipsoid(WGS84Latest))
const NORMALIZEDMINORAXISWGS84 = minoraxis(ellipsoid(WGS84Latest)) |>
x-> uconvert(km, x) |> ustrip |> x-> x/MAJORAXISWGS84
# generic information of all the structures
const DEFAULT_METADATA = ConcreteMetaType(
"datum"=> (WGS84Latest, :datum),
"major_axis"=> (MAJORAXISWGS84, :major_axis),
"units_length"=> (km, :units_length),
"units_angle"=> (deg, :units_angle),
"eccentricity" => (ECCENTRICITY2WGS84, :eccentricity2),
"normalized_minor_axis" => (NORMALIZEDMINORAXISWGS84, :normalized_minor_axis),
)
@enum INTENT INPUT=1 OUTPUT=2 INPUTOUTPUT=0
# individual information of the columns
const DEFAULT_COLMETADATA = ConcreteMetaType(
"has_dimensions"=> (true, :bool),
"is_normalized" => (false,:bool),
"unit" => (km,:unit),
"intent" => (INPUT,:intent),
)
GeoArray(data; metadata=DEFAULT_METADATA, colmetadata=DEFAULT_COLMETADATA) = GeoArray{eltype(data),ndims(data)}(data, metadata, colmetadata)
# generic structure for ray tracing (as an example)
struct Ray2{T}
point_x::T
point_y::T
direction_x::T
direction_y::T
end
# the data structure
ga=GeoArray(Array{Ray2{Float64},1}(undef,100))
metadata(ga)
```
the idea is that I can keep in the structure some data that are normalized for computational purposes but can be seen in the unit of interests when required for visualization and plotting data. Plus creating a new concrete type is almost painless (the few lines at the beginning) while keeping all the functionality and the modularity of the type
---
### Author:  [@aplavin](https://discourse.julialang.org/u/aplavin)
#### Post date: [April 17, 2025, 9:37pm UTC](https://discourse.julialang.org/t/ann-abstractmetaarrays-jl-a-framework-for-metadata-aware-arrays/128123/4 "2025-04-17T21:37:28Z")
I see, probably it’s indeed different…
Btw,
> [@marco\_menarini](#):
>
> The other thing is that `MetadataArrays.jl` does not support colmetadata arrays.
note that one can use `colmetadata` when MetadataArrays are columns in a StructArray.
---
### Author:  [@marco\_menarini](https://discourse.julialang.org/u/marco_menarini)
#### Post date: [April 17, 2025, 9:50pm UTC](https://discourse.julialang.org/t/ann-abstractmetaarrays-jl-a-framework-for-metadata-aware-arrays/128123/5 "2025-04-17T21:50:23Z")
> [@marco\_menarini](#):
>
> MetadataArrays.jl
That wasn’t particularly clear to me: I saw that struct array can support colmetadata, if the underlying array already supports colmetadata, while MetadataArrays does not even export the functions in their code.
But the main thing was to make something that was more modular, and I could use and adapt for different projects easily: if I just need an array with everything I can use the SimpleMetaArray that has all writing and reading functionalities, if I need something more niche I can simply make something new in few lines that keeps all the basic functions I need