[ANN] DynamicQuantities.jl: type stable physical quantities

image

Happy to share DynamicQuantities.jl, a library that defines a simple, statically-typed Quantity object for working with physical units in Julia.

Dev Build Status Coverage

Physical quantities are stored as values, as opposed to the parametrically-typed units in Unitful.jl. This is done to allow for calculations where physical dimensions are not inferrable at compile time.

Performance

These type-stable quantities can greatly outperform those in Unitful when the compiler cannot infer dimensions:

julia> using BenchmarkTools, DynamicQuantities; import Unitful

julia> dynamic_q = Quantity(0.2, mass=1, length=0.5, amount=3)
0.2 𝐋 ¹ᐟ² 𝐌 ¹ 𝐍 ³

julia> unitful = convert(Unitful.Quantity, dynamic_q)
0.2 kg m¹ᐟ² mol³

julia> f(x, i) = x ^ i * 0.3;

julia> @btime f($dynamic_q, i) setup=(i=rand(1:10));
  9.384 ns (0 allocations: 0 bytes)

julia> @btime f($unitful, i) setup=(i=rand(1:10));
  29.667 μs (42 allocations: 1.91 KiB)

Note the μ and n: this example gets a >3000x speedup. DynamicQuantities allows the compiler to build a function that is type stable, while the Unitful quantity object, which stores its dimensions in the type, requires type inference at runtime.

However, note if the dimensions in your function can be inferred by the compiler, then you can get better speeds with Unitful:

julia> g(x) = x ^ 2 * 0.3;

julia> @btime g($dynamic_q);
  6.083 ns (0 allocations: 0 bytes)

julia> @btime g($unitful);
  1.958 ns (0 allocations: 0 bytes)

While both of these are type stable, because Unitful parametrizes the type on the dimensions, functions can specialize to units and the compiler can optimize away units from the code entirely.

Usage

You can create a Quantity object with a value and keyword arguments for the powers of the physical dimensions (mass, length, time, current, temperature, luminosity, amount):

julia> x = Quantity(0.3, mass=1, length=0.5)
0.3 𝐋 ¹ᐟ² 𝐌 ¹

Elementary calculations with +, -, *, /, ^, sqrt, cbrt, abs are supported, letting you perform calculations on quantities just as you would with Unitful.

julia> x ^ 1.5
0.1643167672515498 𝐋 ³ᐟ⁴ 𝐌 ³ᐟ²
julia> x * Quantity(10.2, mass=2, time=-2)
3.0599999999999996 𝐋 ¹ᐟ² 𝐌 ³ 𝐓 ⁻²
julia> x ^ 3
0.027 𝐋 ³ᐟ² 𝐌 ³
julia> x ^ -1
3.3333333333333335 𝐋 ⁻¹ᐟ² 𝐌 ⁻¹
julia> sqrt(x)
0.5477225575051661 𝐋 ¹ᐟ⁴ 𝐌 ¹ᐟ²

Each return value has the same type as x, which means no type inference is needed.

The dimensions of a Quantity can be accessed either with dimension(quantity) for the entire Dimensions object:

julia> dimension(x)
𝐋 ¹ᐟ² 𝐌 ¹

or with umass, ulength, etc., for specific dimensions. You can strip dimensions with ustrip. You can check for dimensional analysis errors with valid(x) - avoiding a try/catch.

Units

DynamicQuantities works with quantities which store physical dimensions (think of this as using the SI standard units — and only those) and a value, and does not directly provide a unit system. However, DynamicQuantities has an interface with Unitful: you can use Unitful to parse units, and then use the DynamicQuantities->Unitful extension for conversion:

julia> using Unitful: Unitful, @u_str

julia> x = 0.5u"km/s"
0.5 km s⁻¹

julia> y = convert(DynamicQuantities.Quantity, x)  # Auto reduces to SI
500.0 𝐋 ¹ 𝐓 ⁻¹

julia> y2 = y^2 * 0.3
75000.0 𝐋 ² 𝐓 ⁻²

julia> x2 = convert(Unitful.Quantity, y2)  # Back to SI
75000.0 m² s⁻²

julia> x^2*0.3 == x2  # Same as if we did it in Unitful
true

This means you could use Unitful as a user interface, for taking inputs from its vast catalog of physical units, and then DynamicQuantities.jl for type-stable calculations where needed.

Vectors

Because the types are stable you can have mixed units in a vector:

julia> v = [Quantity(randn(), mass=rand(0:5), length=rand(0:5)) for _=1:5]
5-element Vector{Quantity{Float64}}:
 2.2054411324716865 𝐌 ³
 -0.01603602425887379 𝐋 ⁴ 𝐌 ³
 1.4388184352393647 
 2.382303019892503 𝐋 ² 𝐌 ¹
 0.6071392594021706 𝐋 ⁴ 𝐌 ⁴

Internals

DynamicQuantities.jl is only ~300 lines of code. The main two types, Dimensions, and Quantities, are simple structs:

import Ratios: SimpleRatio

const R = SimpleRatio{Int}  # Faster Rational{Int}

struct Dimensions
    length::R
    mass::R
    time::R
    current::R
    temperature::R
    luminosity::R
    amount::R
end
struct Quantity{T}
    value::T
    dimensions::Dimensions
    valid::Bool
end

The fields of Dimensions store the powers of each physical dimension of the 7 used in the SI system.

The rest of the library is just setting up helper utilities and overloading math operators.


The library resulted from encouragement from others in this discussion. I thought I’d share it broadly now that it seems to be working. Try it out with

] add https://github.com/SymbolicML/DynamicQuantities.jl

(Pkg registration t-3 days)

I want to also warmly welcome contributors to this package. These sorts of units packages seem pretty general so I am happy to consider suggestions/PRs.

52 Likes

Very cool!!!

2 Likes

Thank you!

1 Like

I realized this will also helps a lot with precompilation, as you only have to compile against a single Quantity{T}, rather than any conceivable combination of dimensions a user might pass in.

2 Likes

This is very neat. I love seeing more approaches to these kind of fundamentals, and I think this approach of encoding the dimensions but not units in the value and not the type of a struct makes a lot of sense for a wide variety of use-cases.

Quick note on performance: since Unitful quantities are represented more compactly in memory than DynamicQuantities quantities, performance measures do tip more in Unitful’s favor when benchmarking operations on densely packed vectors of homogenous quantities:

julia> using BenchmarkTools, DynamicQuantities; import Unitful

julia> dynamic_q = Quantity.(rand(Float64, 2500), mass=1, length=0.5, amount=3);

julia> unitful = convert.(Unitful.Quantity, dynamic_q);

julia> f(x, i) = x ^ i * 0.3;

julia> @btime f.($dynamic_q, i) setup=(i=rand(1:10));
  47.400 μs (2 allocations: 312.55 KiB)

julia> @btime f.($unitful, i) setup=(i=rand(1:10));
  69.147 ms (105009 allocations: 4.67 MiB)

julia> using Unitful: @u_str

julia> 69.147u"ms" / 47.400u"μs" |> Unitful.upreferred
1458.7974683544305

julia> g(x) = x ^ 2 * 0.3;

julia> @btime g.($dynamic_q);
  45.301 μs (2 allocations: 312.55 KiB)

julia> @btime g.($unitful);
  1.489 μs (2 allocations: 19.61 KiB)

julia> 45.301u"μs" / 1.489u"μs" |> Unitful.upreferred
30.42377434519812

So I’m seeing an advantage of roughly 1,500x for DynamicQuantities for the non-inferred dimension case but an advantage of 30x for Unitful when the dimensions can be inferred.

Interestingly, operations on heterogeneous arrays only have a 27x advantage for DynamicQuantities for g. I was expecting something more similar to the 1,500x gap for the non-inferred homogeneous case.

julia> dq = Quantity.(rand(Float64, 2500), mass=1, length=0.5, amount=3).^rand(1:10,2500);

julia> uq = convert.(Unitful.Quantity, dq);

julia> @btime g.($dq);
  44.200 μs (2 allocations: 312.55 KiB)

julia> @btime g.($uq);
  1.191 ms (7015 allocations: 149.33 KiB)

julia> 1.191u"ms" / 44.200u"μs" |> Unitful.upreferred
26.945701357466064

julia> @btime f.($dq, i) setup=(i=rand(1:10));
  46.769 μs (2 allocations: 312.55 KiB)

julia> @btime f.($uq, i) setup=(i=rand(1:10));
  78.515 ms (109513 allocations: 4.76 MiB)

julia> 78.515u"ms" / 46.769u"μs" |> Unitful.upreferred
1678.7829545211573
4 Likes

Great to have dynamic alternative to Unitful, for those (rarer but existing) cases when typed units are inefficient. Can it be made a drop-in replacement to Unitful? So that changing a single using line switches between typed and dynamic unit implementation.

3 Likes

In general I think this would be best done with StructArrays.jl, so that the dimension-specific operations can be vectorized too. I’d be very curious to see how that changes your measurement.

Perhaps this is also because the dimensions are stored as SimpleRatio{Int} rather than an Int8 (which is more amenable to vectorization)? @Oscar_Smith was suggesting this in the initial thread based on some measurements:

It’s likely that few people need more than an Int8 for physical dimensions. The tricky part is dealing with integer overflows if we stick with SimpleRatio: even four repeated + (2//3) - (2//3) would be enough to overflow Int8. (Since SimpleRatio doesn’t repeatedly divide by the gcd)

@Oscar_Smith One other idea could be to create a lookup table for gcd(::Int8, ::Int8). This would only be 65 kB (256^2) to store. That would probably give a big speedup for regular Rational{Int8} and perhaps make it practical to use here.

(i.e., you could have a mapping from (::Int8, ::Int8) -> reduced_form for rational numbers, and use it as post-processing on every SimpleRatio{Int8} calculation)

It might be worthwhile to define methods implementing the AbstractArray interface as well as the broadcasting interface for ::Quantity{<:AbstractArray}. Or if there are legitimate use-cases where Quantity{<:AbstractArray} should be treated like an array that has dimensions rather than an array where each value has dimensions, you could define a new QuantityArray wrapper type implementing those interfaces. Actually a new type would probably be better since you can’t do Quantity{<:AbstractArray{T,N}} <: AbstractArray{Quantity{T},N}.

That would give you probably 99% of the performance advantage of Unitful for homogeneous arrays being operated on by fully-inferred functions without adding too much additional complexity.

3 Likes

@aplavin I think the proposal here by @j-fu: Propose some functions and macros for unitful integration. by j-fu · Pull Request #11 · SymbolicML/DynamicQuantities.jl · GitHub
might address part of this. I’m not sure if the best strategy would be to have a separate @u_str, or use a @q_str as is proposed here.

1 Like

In the context of that PR I see Unitful as a resource of truth on what is km, Hz etc which needs to be tapped for making DynamicQuantites fully functional. OTOH, due the structural differences in the way things are handeled type-wise it is hard for me to see if drop-in replacement can work. In the moment I think that the optimal solution indeed would consist in both packages working together.

1 Like

I’m starting to think so too. There are so many physical constants and units that Unitful.jl (and its extended family of packages) export; it seems like so much effort for little gain to need to re-implement them all.

I’ve been trying out Unitful → DynamicQuantities for my own stuff and it seems to work well. In SymbolicRegression.jl my design is for the user to call, e.g.,

dataset = Dataset(X, y; variable_units=[u"km", u"A", u"s^2"])

then this vector would be immediately parsed and converted to DynamicQuantities.jl, where it is type stable and can be used for downstream processing/dimensional analysis.

The bonus is that the user could pass physical constants as well. e.g., UnitfulAstro.jl has most of the stuff my research would use, and it would get consistently mapped into a DynamicQuantities.Quantity object for processing.

Do you think you could draft a PR on this? I don’t completely follow how this would work but I definitely agree this is important to add.

Also

Are there any proposals to make the abstract array interface a trait rather than a type? (Or maybe that would break Julia…)

I wonder if it might be worth it to define, e.g,

const m = Quantity(1.0, length=1)
const g = Quantity(1e-3, mass=1)
const s = Quantity(1.0, time=1)
…
@add_prefixes m
@add_prefixes g
…

Along with a @u_str and uparse function. And a set of other common units.

Then people don’t need to use the Unitful extension for units.


Edit: here’s an implementation:

macro add_prefixes(base_unit, prefixes)
    @assert prefixes.head == :tuple
    expr = _add_prefixes(base_unit, prefixes.args)
    return expr |> esc
end

function _add_prefixes(base_unit::Symbol, prefixes)
    all_prefixes = (f=1e-15, p=1e-12, n=1e-9, μ=1e-6, u=1e-6, m=1e-3, c=1e-2, k=1e3, M=1e6, G=1e9, T=1e12, P=1e15)
    expr = Expr(:block)
    for (prefix, value) in zip(keys(all_prefixes), values(all_prefixes))
        prefix in prefixes || continue
        new_unit = Symbol(prefix, base_unit)
        push!(expr.args, :(const $new_unit = $value * $base_unit))
    end
    return expr
end

# SI base units
const m = Quantity(1.0, length=1)
const g = Quantity(1e-3, mass=1)
const s = Quantity(1.0, time=1)
const A = Quantity(1.0, current=1)
const K = Quantity(1.0, temperature=1)
const cd = Quantity(1.0, luminosity=1)
const mol = Quantity(1.0, amount=1)

@add_prefixes m (f, p, n, μ, u, c, m, k, M, G)
@add_prefixes g (μ, u, m, k)
@add_prefixes s (f, p, n, μ, u, m)
@add_prefixes A (n, μ, u, m, k)
@add_prefixes K (m,)
@add_prefixes cd (m,)
@add_prefixes mol (m,)

# SI derived units
const Hz = inv(s)
const N = kg * m / s^2
const Pa = N / m^2
const J = N * m
const W = J / s
const C = A * s
const V = W / A
const F = C / V
const Ω = V / A
const T = N / (A * m)

@add_prefixes Hz (k, M, G)
@add_prefixes N ()
@add_prefixes P ()
@add_prefixes J ()
@add_prefixes W (k, M, G)
@add_prefixes C ()
@add_prefixes V (m, k, M, G)
@add_prefixes F ()
@add_prefixes Ω ()

# Do not wish to define Gaussian units, as it changes
# some formulas. Safer to force user to work exclusively in one unit system.

"""
    uparse(s::AbstractString)

Parse a string containing an expression of units and return the
corresponding `Quantity` object. For example, `uparse("m/s")`
would be parsed to `Quantity(1.0, length=1, time=-1)`.
"""
function uparse(s::AbstractString)
    return eval(Meta.parse(s))::Quantity{DEFAULT_VALUE_TYPE,DEFAULT_DIM_TYPE}
end

macro u_str(s)
    return esc(uparse(s))
end

I think that’s pretty much everything someone would need in the SI system.

I could also make it so that the printout is m kg s, rather than L M T as is done currently, so its more obvious what is assumed.

Example:

julia> x = 1.0u"km/s^2"
1000.0 m s⁻²

Edit 2: PR submitted for this here: Create in-house units module by MilesCranmer · Pull Request #22 · SymbolicML/DynamicQuantities.jl · GitHub

3 Likes

Looks neat. One question:

I would guess that most (all?) units appear as fractions (including integers).

  • would it be possible to specify the length unit as length=1//2?

yeah. that works.

I haven’t tested the DynamicQuantities package yet. To me, it looks like a system for assigning dimensions to quantities in the sense of Lin and Segel’s classic book [ “Mathematics Applied to Deterministic Problems in the Natural Sciences” (Classics in Applied Mathematics, Series Number 1), 1st ed.]. As far as I remember, Lin and Segel discuss Buckingham’s PI theorem and dimensionless groups.

Stichlmair, J. G. (2002) [“Scale-up Engineering, Begell House, Inc., New York. ISBN 1-56700-160-2”] contains a number of practical examples on dimensionless groups in some 150-200 pages (??), mainly from chemical engineering.

The key idea of Buckingham’s PI theorem can be described simply in some 5 pages. The dimensionless groups are easily found from the nullspace of a matrix that can be set up from the list of quantities and the exponents of the various dimensions (e.g., length = 2, etc.). So this is straightforward linear algebra.

Of course, the vectors in the nullspace of a matrix are not unique, so the found dimensionless groups are not unique. Perhaps there is some theory on making these unique? E.g., that the elements of the nullspace should be integers or rational numbers, perhaps with a minimal norm, or something like that. The dimensionless numbers that show up in physics typically also have some physical interpretation. Don’t know how easy it is to automate the discovery of these, though.

The possibility of checking dimensional consistency in equations is very useful. Perhaps this can also help promote the idea of using dimensionless groups in arguments of transcendental functions, etc… makes life much easier when changing units in models…

Questions:

  • Does DynamicQuantities.jl have support for finding dimensionless groups for a collection of quantities? [Could be useful to reduce the number of variables in regression?]
  • Could it be possible to also find typical dimensionless groups such as those of physics? [Reynolds number, Prandtl number, etc., etc.]

Checking for dimensional consistency in equations is one thing. Checking that units are consistent is also very important.

Anyways, looks like a neat package.

I’ve been thinking about this, and I don’t think it’s as trivial as I originally thought. You can pretty easily define the struct below and fill in the required methods for the array interface:

struct HomogeneousQuantityArray{T, R, N, V<:AbstractArray{T,N}} <: AbstractArray{Quantity{T,R}, N}
    values::V
    dimensions::Dimensions{R}
end

And that gets you savings w.r.t. array-size, but unless you define most of the <:Number interface for ::Dimensions, there’s no way to calculate what the dimensions field should be after a broadcast without performing the dimension calculation individually for each array element.

If you did define the number interface for ::Dimensions, you’d basically override Broadcast.broadcasted to first try the broadcasted function on the dimension field and only proceed to the values field if no error (e.g. from a call to ^(::Any, ::Dimension)).

The problem is all the functions that dispatch on ::AbstractArray. Not sure if anyone has written up a proposal for Julia 2.0.

1 Like

UnitfulBuckinghamPi.jl possibly does what you suggest.

2 Likes

Here’s a few updates/answers:

1. Units

The following PR implements various SI base and derived units inside DynamicQuantities.jl: Create in-house units module by MilesCranmer · Pull Request #22 · SymbolicML/DynamicQuantities.jl · GitHub. With this you don’t need to rely on Unitful.jl anymore.

2. Abstract types

There is a PR for abstract super types here: Make `AbstractQuantity` and `AbstractDimensions` by MilesCranmer · Pull Request #24 · SymbolicML/DynamicQuantities.jl · GitHub. I wonder if that makes it easier at all to make an array type @non-Jedi

3. Dimensional analysis

@BLI The PR here: Dimensional constraints by MilesCranmer · Pull Request #220 · MilesCranmer/SymbolicRegression.jl · GitHub might be of interest. It implements a dimensional analysis routine in SymbolicRegression, using DynamicQuantities, to search for expressions which are dimensionally valid and optimize some objective. The tricky part was dealing with free constants that can take on an arbitrary dimension — that is implemented as WildcardQuantity in that PR which does this automatically by basically propagating a wildcard dimension through an expression evaluation, which can cancel dimensions when needed (consuming itself in the process).

5 Likes