[ANN] Covar.jl - Propagation of Covariant Uncertainties

Package Announcements
,
1

Covar.jl is a package to handle systems with multiple uncertainties, whose correlations are described by covariance matrices. Part of the motivation for developing this package was the desire to nicely handle fit results, as the resulting fit parameters are usually not independent. For independent measurements, Measurements.jl already exists, but it doesn’t have a way to handle covariance matrices, so the decision was made to develop this package, as the underlying structure had to be quiet different from Measurements.jl.

The package makes heavy use of Julia’s method overloading and DiffRules.jl, in order to make error propagation as intuitive as possible. A CovariantSystem can be conveniently constructed with the @covar_system macro, which returns an array of CovariantVars. Any mathematical operation on these produces a DerivedVar, which handles the error propagation.

A simple example:

julia> a, b = @covar_system [2., 3.] [.3 .1; .1 .4]
2-element Array{CovariantVar{Float64},1}:
 x_1 = 2.0 ± 0.5477225575051661
  Vₘ = [0.3 0.1; 0.1 0.4]
 x_2 = 3.0 ± 0.6324555320336759
  Vₘ = [0.3 0.1; 0.1 0.4]

julia> c = a^2 + 2b + 1
f({xᵢ}) = 11.0 ± 2.8284271247461903
  {∇⃗f} = Array{Float64,1}[[4.0, 2.0]]
  {Vₘ} = Array{Float64,2}[[0.3 0.1; 0.1 0.4]]

julia> val(c), err(c)
(11.0, 2.8284271247461903)

The package is still in ongoing development, but should already be quite usable. There’s still work to be done, especially adding tests. I’m open for issues or suggestions, as I’m still fairly new to the language.

6 Likes
2

Internals of Measurements.jl aren’t set in stone :wink: python package uncertainties supports both quantity correlated through a function and by a covariance matrix, so it isn’t impossible to do them both.

I’d be happy to discuss with you what to do to integrate covariance matrices in Measurements.jl :slight_smile:

3 Likes
3

That would definitely make sense to combine the two. Technically you can already do uncertainties without covariance by using diagonal covariance matrices or a different system for each variable, but that’s not really convenient and probably comes with at least some overhead. It would probably be nice to have Measurement be an abstract type, that can either be an independent measurement or one with covariance. I see you’re using Calculus.jl for derivatives, I’m using DiffRules right now, because it seems pretty lightweight, and gives a nice way to generate methods for every operator with metaprogramming. I’m also not sure how to handle complex measurements. It seems like you just handle the real and imaginary part as two independent measurements, but that could be quite complicated if you want to allow covariance between these two. Other than that, I think it would be great to have a package to handle both cases, it just seems a little bit daunting to me, as Measurements.jl already brings with it quite some complexity.

1 Like
4

Gotta try this with DiffEq :slight_smile:

5

I’ve already experimented with a Measurement abstract type and two different concrete types for independent quantities and derived quantities (see Proposal: new types for independent measurements and dependent measurements · Issue #7 · JuliaPhysics/Measurements.jl · GitHub), but that wasn’t really good, because arrays of measurements would be inefficient being the union of two different concrete types (this was a couple of years ago, maybe now the situation would be better?). Most annoyingly, the two different types prevented some mathematical operations to work out-of-the-box, because that broke the assumption that float(T<:AbstractFloat) = T.

Not really, Calculus.jl is used only to allow for numerically computing the derivatives of arbitrary functions for which there is no known analytic derivative, of for functions based on C/Fortran ccalls. The derivatives of known functions are manually coded in the package. When I wrote the package I found that the performance in this way was much better than using the AD packages then available.

Not at all. Correlation is correctly treated out-of-the-box, thanks to the fact that all mathematical operations of complex arguments in Julia’s Base are implemented in terms of elementary operations on the real and imaginary parts (see more efficient type · Issue #1 · JuliaPhysics/Measurements.jl · GitHub). Complex quantities in the test suite do have correlated real and imaginary parts, see u and v in https://github.com/JuliaPhysics/Measurements.jl/blob/95087d7dea1936586d449ef6d6e9808074e0eff1/test/complex.jl

It probably is quite complex at this point, but nothing prevents us from improving it :slight_smile: The good point is that Measurements.jl has an extensive test suite, thus it isn’t difficult to rewrite internals while keeping in the end the same functionalities as before, the difficult part is to devise how to rewrite the internals :sweat_smile:

2 Likes