How to add dimensions in FlexUnits

I would like to add a dimension (say, for example, Money) in FlexUnits, and define a base unit for it (say EUR).
I have searched in the doc and online, but I can’t seem to find how to do that. Could someone point me to a tutorial?

In case it clarifies what I am trying to do, here is what I am using with Unitful:

module MyUnits
using Unitful
@dimension MU "MU" MonetaryUnit false
@refunit EUR "EUR" Euro MU true
end
Unitful.register(MyUnits)
using .MyUnits

# I can then do things like
cu = 1u"EUR/L"
c = cu * 10u"m^3"  |> u"kEUR"

Adding a new fundamental dimension in FlexUnits is a bit more involved than in Unitful, because you need to maintain your own unit registry. Here is a complete example that you can copy directly to add a currency dimension with EUR as the base unit:

module MyUnits

using FlexUnits
using FlexUnits.RegistryTools

@kwdef struct MoneyDimensions{P} <: AbstractDimensions{P}
    m  ::P = zero(FixRat32)
    kg ::P = zero(FixRat32)
    s  ::P = zero(FixRat32)
    A  ::P = zero(FixRat32)
    K  ::P = zero(FixRat32)
    cd ::P = zero(FixRat32)
    mol::P = zero(FixRat32)
    EUR::P = zero(FixRat32)
end
MoneyDimensions(args::Real...) = MoneyDimensions{FixRat32}(args...)

const UNITS = PermanentDict{Symbol,Units{MoneyDimensions{FixRat32},AffineTransform{Float64}}}()

registry_defaults!(UNITS)
register_unit!(UNITS, "EUR" => MoneyDimensions{FixRat32}(EUR=1))
register_unit!(UNITS, "kEUR" => 1000*UNITS[:EUR])

uparse(str::String) = RegistryTools.uparse(str, UNITS)
qparse(str::String) = RegistryTools.qparse(str, UNITS)

macro u_str(str); return suparse_expr(str, UNITS); end
macro ud_str(str); return uparse_expr(str, UNITS); end
macro q_str(str); return qparse_expr(str, UNITS); end
macro U_str(str); suexpr = suparse_expr(str, UNITS); return :($typeof($suexpr)); end
macro D_str(str); suexpr = suparse_expr(str, UNITS); return :($dimtype($suexpr)); end

utype() = RegistryTools.regunittype(UNITS)
dtype() = RegistryTools.regdimtype(UNITS)

export @u_str, @ud_str, @q_str, @U_str, @D_str, uparse, qparse, utype, dtype

end

Then, in your main module, you only need

using FlexUnits
using .MyUnits

and you can use all units registered by registry_defaults!, together with the additional currency units, for example 1u"m", 1u"kW", and 1u"EUR".

You do not need to, and generally should not, also using FlexUnits.UnitRegistry. FlexUnits.UnitRegistry exports another set of macros with the same names, such as @u_str and @ud_str, but those macros are bound to FlexUnits’ default UnitRegistry.UNITS, not to MyUnits.UNITS. Bringing both registries into the same namespace can therefore cause name conflicts, and the default registry does not contain the EUR dimension anyway.

One more thing if you plan to distribute your package for others to use.

I would avoid re-exporting the custom @u_str, @ud_str, etc. from your package. Otherwise users may run into conflicts if they also do

using FlexUnits.UnitRegistry

because the two registries export macros with the same names.

There is also a deeper issue. Units from the two registries use different dimension types:

MyUnits.u"m"
# Units{MoneyDimensions{FixRat32}, ...}

FlexUnits.UnitRegistry.u"m"
# Units{Dimensions{FixRat32}, ...}

So even though both are metres, they don’t directly interoperate for arithmetic or unit conversion.

A short-term workaround is to add a Dimensions -> MoneyDimensions promotion/conversion bridge. But long term, I think this is something FlexUnits could handle more cleanly, for example by making it easier to add a new fundamental dimension without having to copy and maintain a whole separate registry.

Maybe @Deduction42 has some thoughts on this?

Excellent, that is exactly what I needed, and it does work. Thank you!
The issue about re-exporting is not relevant to me at this time, but thanks for the warning.

I think I found an issue with the proposed solution. When trying to use simplify, I get an error saying “type does not have a definite number of fields” (full message below). To make sure, I use simplify on a standard unit, not a customized one. The simplification does work if I stick to the default .UnitRegistry module. @karei, was that expected?

julia> 1u"Pa"
1.0 kg/(m s²)

julia> 1u"Pa" |> simplify
ERROR: ArgumentError: type does not have a definite number of fields
Stacktrace:
  [1] fieldcount
    @ .\runtime_internals.jl:1163 [inlined]
  [2] fieldnames(t::DataType)
    @ Base .\runtime_internals.jl:336
  [3] static_fieldnames(t::Type)
    @ FlexUnits C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\types.jl:64
  [4] dimension_names
    @ C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\types.jl:238 [inlined]
  [5] map_dimensions
    @ C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\math.jl:12 [inlined]
  [6] raw_div(d1::Main.MyUnits.MoneyDimensions{FixRat32}, d2::Dimensions{FixRat32})
    @ FlexUnits C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\math.jl:41
  [7] /
    @ C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\math.jl:65 [inlined]
  [8] (::FlexUnits.var"#fit_improvement#77"{Main.MyUnits.MoneyDimensions{…}, Units{…}})(p::Int64)
    @ FlexUnits C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\utils.jl:375
  [9] _optimal_unit_fit(d::Main.MyUnits.MoneyDimensions{…}, u::Units{…}; maxiter::Int64)
    @ FlexUnits C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\utils.jl:378
 [10] _optimal_unit_fit
    @ C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\utils.jl:372 [inlined]
 [11] #_unit_power_simplify!##0
    @ C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\utils.jl:349 [inlined]
 [12] iterate
    @ .\generator.jl:48 [inlined]
 [13] _collect(c::Vector{…}, itr::Base.Generator{…}, ::Base.EltypeUnknown, isz::Base.HasShape{…})
    @ Base .\array.jl:810
 [14] collect_similar
    @ .\array.jl:732 [inlined]
 [15] map
    @ .\abstractarray.jl:3375 [inlined]
 [16] _unit_power_simplify!(numervec::Vector{…}, denomvec::Vector{…}, dref::Main.MyUnits.MoneyDimensions{…}, unit_set::Vector{…})
    @ FlexUnits C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\utils.jl:349
 [17] simplify(dref::Main.MyUnits.MoneyDimensions{FixRat32}, unit_set::Vector{Units{Dimensions{…}, AffineTransform{…}}})
    @ FlexUnits C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\utils.jl:279
 [18] simplify(dref::StaticDims{kg/(m s²)}, unit_set::Vector{Units{Dimensions{FixRat32}, AffineTransform{Float64}}})
    @ FlexUnits C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\utils.jl:264
 [19] simplify(q::Quantity{Float64, StaticDims{kg/(m s²)}}, unit_set::Vector{Units{Dimensions{…}, AffineTransform{…}}})
    @ FlexUnits C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\utils.jl:254
 [20] simplify(q::Quantity{Float64, StaticDims{kg/(m s²)}})
    @ FlexUnits C:\Users\Me\.julia\packages\FlexUnits\f0keH\src\utils.jl:215
 [21] |>(x::Quantity{Float64, StaticDims{kg/(m s²)}}, f::typeof(simplify))
    @ Base .\operators.jl:972
 [22] top-level scope
    @ REPL[1]:1

This is because simplify doesn’t look up Pa in the MyUnits.UNITS. It falls back to FlexUnits’ global preferred unit set instead:

https://github.com/Deduction42/FlexUnits.jl/blob/4d572099a2fc935883fd92d960bf2dcc0516af2b/src/utils.jl#L212

Those units use the built-in Dimensions type, so they aren’t compatible with MoneyDimensions.

As a workaround, you can define your own preferred unit set by overloading preferred_units:

FlexUnits.preferred_units(::Type{<:MoneyDimensions}) = [MyUnits.UNITS[:F], 
MyUnits.UNITS[:H], MyUnits.UNITS[:T], MyUnits.UNITS[:Ω], MyUnits.UNITS[:V], 
MyUnits.UNITS[:W], MyUnits.UNITS[:J], MyUnits.UNITS[:Pa], MyUnits.UNITS[:N], 
MyUnits.UNITS[:C], MyUnits.UNITS[:L], MyUnits.UNITS[:EUR]]

1u"Pa" # 1.0 kg/(m s²)
1u"Pa" |> simplify # 1.0 Pa

You can put this overload directly inside the MyUnits module.

Thanks, that works!

Thank you so much for replying on my behalf. This is exactly what I would have recommended. I will indeed add this example. I was planning on making something like this for angles, but this works great too!

I did think about working with different Dimensions types, but I haven’t found a very elegant solution other than promotion, where you have to write those promotion and conversion rules by hand. What makes things more difficult is that you cannot export string macros and parsers from more than one registry at a time (as you mentioned). This means that you will need to fully state the module name before the macro (which you did).

Because of this, I find that it’s just easier to build a new dimension definition that is a superset of the old one, and build a while new registry from that (which is what you did in your example). This is why I created the registry_defaults! function which you masterfully used. That way you have a nice registry with all your new dimensions, and parsing units from strings is type-stable and you can export macros making this registry your source of truth. I can’t think of a cleaner way to do this.

To be fair, the dimension object design philosophy in FlexUnits resembles DynamicQuantities.jl much more than Uniftul.jl under the hood. This is because fast dynamic inference (when units are unknown at compile time) and reasonable bounds on the number of types were two key requirements when I built this package. Unfortunately, open-ended dimensions in Unitful contribute to a huge proliferation of types and significant dynamic inference costs.

Because of this, I simply accepted that I’ll need to define new registries for new dimension types, so I tried to make the process as painless as possible. Trying to do this in DynamicQuantities.jl is very painful (I learned this by trying to support affine units, and that whole ordeal was one of the main reasons why I built this package).

I now added a slightly more refined example on how to do this in the documentation. It also includes additional steps for making simplify work and the promotion/conversion rules you need to add in order to combine operations with the original Dimensions types.

I’ve been pretty busy the last couple of days, so I’m only just getting back to this now.

Long story short, for a package that puts a lot of emphasis on the performance of dynamic dimension operations, I also think the current design is probably the right one: users define a new unit registry and carry over the existing seven dimensions from FlexUnits.

What I would suggest is providing a dedicated macro for registering a new dimension type and letting it take care of all the repetitive boilerplate for the user, such as promote, convert, registry_defaults!, and so on. That would not only make it easier to extend the dimension system, but also allow the new dimensions to work seamlessly with the existing Dimensions type.

I’ve drafted a rough proposal below for reference:

function _check_dimension_extension(::Type{New}, ::Type{Old}) where {New<:AbstractDimensions,Old<:AbstractDimensions}
    new_names, old_names = dimension_names(New), dimension_names(Old)
    missing_dims = filter(name -> name ∉ new_names, old_names)
    !isempty(missing_dims) && throw(ArgumentError("$(New) cannot extend $(Old): missing dimensions $(missing_dims)"))
    nothing
end

@generated function _convert_dimension_extension(::Type{New}, d::Old) where {P,New<:AbstractDimensions{P},Old<:AbstractDimensions}
    new_names, old_names = dimension_names(New), dimension_names(Old)
    missing_dims = filter(name -> name ∉ new_names, old_names)
    !isempty(missing_dims) && error("$(New) cannot extend $(Old): missing dimensions $(missing_dims)")
    args = map(new_names) do name
        name ∈ old_names ? :(convert($P, getproperty(d, $(QuoteNode(name))))) : :(zero($P))
    end
    return :(isunknown(d) ? unknown($New) : $New($(args...)))
end

macro register_dimensions(units, new, old)
    return esc(quote
        FlexUnits.RegistryTools._check_dimension_extension($new, $old)

        @inline $new(args::Real...) = $new{FlexUnits.FixRat32}(args...)

        @inline Base.promote_rule(::Type{$new{P1}}, ::Type{$new{P2}}) where {P1,P2} =
            $new{promote_type(P1, P2)}
        @inline Base.promote_rule(::Type{$new{P1}}, ::Type{$old{P2}}) where {P1,P2} =
            $new{promote_type(P1, P2)}
        @inline Base.promote_rule(::Type{$old{P1}}, ::Type{$new{P2}}) where {P1,P2} =
            $new{promote_type(P1, P2)}

        @inline (::Type{$new{P}})(d::$old) where P =
            FlexUnits.RegistryTools._convert_dimension_extension($new{P}, d)

        FlexUnits.RegistryTools.registry_defaults!(units)

        uparse(str::String) = FlexUnits.RegistryTools.uparse(str, units)
        qparse(str::String) = FlexUnits.RegistryTools.qparse(str, units)

        macro u_str(str); return FlexUnits.RegistryTools.suparse_expr(str, units); end
        macro ud_str(str); return FlexUnits.RegistryTools.uparse_expr(str, units); end
        macro q_str(str); return FlexUnits.RegistryTools.qparse_expr(str, units); end
        macro U_str(str); suexpr = FlexUnits.RegistryTools.suparse_expr(str, units); return :(typeof($suexpr)); end
        macro D_str(str); suexpr = FlexUnits.RegistryTools.suparse_expr(str, units); return :(FlexUnits.RegistryTools.dimtype($suexpr)); end

        utype() = FlexUnits.RegistryTools.regunittype(units)
        dtype() = FlexUnits.RegistryTools.regdimtype(units)

        nothing
    end)
end

With something like this, defining a new dimension set could become very simple:

module MyUnits

using FlexUnits
using FlexUnits.RegistryTools

@kwdef struct MoneyDimensions{P} <: AbstractDimensions{P}
    # existing 7 dimensions
    EUR::P = zero(FixRat32)
end

const UNITS = PermanentDict{Symbol,Units{MoneyDimensions{FixRat32},AffineTransform{Float64}}}()

@register_dimensions UNITS MoneyDimensions Dimensions

register_unit!(UNITS, "EUR" => MoneyDimensions{FixRat32}(EUR=1))
register_unit!(UNITS, "kEUR" => 1000*UNITS[:EUR])

const CURRENCY_UNITS = [UNITS[k] for k in [:F,:H,:T,:Ω,:V,:W,:J,:Pa,:N,:C,:L,:EUR]]
FlexUnits.preferred_units(::Type{<:MoneyDimensions}) = CURRENCY_UNITS

export @u_str # ...

end

Very true. This has given me a lot to think about. I think I can put even more stuff inside the macro and make sure the “registry_defaults!” looks up the symbols in the base dimensions object and makes sure every dimension symbol itself is registered (I planned to do this now that dimensions uses base unit symbols). Also, you don’t need to add “base dimensions” to the preferred units, those are already assumed. This means that the only thing inside your module needs to be the imports and the MoneyDimensions definition. Everything else can be put inside the macro. Extra unit registrations can be done inside or outside the module; there is no cost to doing it either way (I often give customers options to define their own units in excel and my app registers them at runtime on startup). As long as the units don’t change, anything goes.

I just registered a new version with changes to drastically simplify the process of building new registries.

It turns out that I already had functions for generically converting one dimension to another, but your generated function design was more elegant and produced more informative errors, so I modified my existing function to exhibit your suggested behavior. Promotion rules favoring the dimensional superset were also added so now users don’t need to do this work.

Thanks again for your macro suggestion. I really should have thought of that. I added two macros to RegistryTools to make this currency dimension problem much easier to solve.

using FlexUnits
@kwdef struct MoneyDimensions{P} <: AbstractDimensions{P}
    m   ::P = zero(FixRat32)
    kg  ::P = zero(FixRat32)
    s   ::P = zero(FixRat32)
    A   ::P = zero(FixRat32)
    K   ::P = zero(FixRat32)
    cd  ::P = zero(FixRat32)
    mol ::P = zero(FixRat32)
    €   ::P = zero(FixRat32)
end
MoneyDimensions(args::Real...) = MoneyDimensions{FixRat32}(args...)

module CurrencyUnits
    using FlexUnits.RegistryTools
    import ..MoneyDimensions

    # Define your unit registry dict and auto-populate it
    const UNITS = PermanentDict{Symbol,Units{MoneyDimensions{FixRat32},AffineTransform{Float64}}}()
    registry_defaults!(UNITS) # registry_defaults! is compatible with MoneyDimensions because it is a superset of Dimensions
    register_unit!(UNITS, "EUR" => UNITS[:€]) # additional registrations can happen here

    # Optional support for simplification but recommended if dimensions are not Dimensions{FixRat32}
    const PREFERRED_UNITS = [UNITS[u] for u in [:F, :H, :T, :Ω, :V, :W, :J, :Pa, :N, :C, :L]] # typical simplification basis you can change at will
    @generate_unit_simplifier(PREFERRED_UNITS) # takes care of simplification boilerplate code

    @generate_registry_exports(UNITS) # takes care of typical string macro and unit parsing exports
end

This produces the desired registry and simplification behaviour.

using .CurrencyUnits

julia> electricity_price = 0.195u"€/(kW*hr)"
5.416666666666666e-8 (s² €)/(m² kg)

julia> electricity_price = 0.195u"€/(kW*hr)" |> simplify
5.416666666666666e-8 €/J

It doesn’t get much simpler than that; because it’s so simple, I’ve now included it in the README file too (which needed an upgrade anyway).

These ideas seem to have already been incorporated into the core API in a more elegant way than what I proposed. Very nice design.:slight_smile: