`mergewith` issue with `Unitful`

General Usage
,
1

I’ve been using mergewith(+, …) with Dicts containing Unitful values, and it worked great for a while … until it suddenly didn’t. And slight reorganization of the arguments made it work again, so I’m not quite sure what’s going on, exactly.

A simple example:

struct K
	s
end

mergewith(+,
Dict(
	K("A") => 1,
	"B" => 1g
),
Dict(
	K("A") => 1,
),
Dict(
	K("C") => 1g
))

With this, I get the error

ERROR: promotion of types Quantity{Number, NoDims, Unitful.FreeUnits{(), NoDims,
nothing}} and Int64 failed to change any arguments

I can see how you might get into trouble with two unitless scalars, where one is explicitly annotated as having no type. But with more complicated examples (haven’t as yet got a minimal example for this), I also get stuff like:

DimensionError: MyUnit and 3.9247451750516955 are not dimensionally compatible.

In this example, 3.9247451750516955 actually has the unit MyUnit (in the first of the Dicts being merged).

I would have thought this should have worked (given that the individual values can be added together, according to the dimensions). Could it be that the promotion “magic” of mergewith isn’t quite robust enough? Or am I doing something wrong? (Custom hash/== shouldn’t be needed, I guess; and, indeed, it doesn’t seem to make a difference.)

2

Not directly regarding the errors in the post, but adding scalars with different units producing an error is exactly the kind of logic-checking units are supposed to catch. Can you explain how these dimensionfull and dimensionless additions came about? Perhaps it is better to fix the original ‘sin’ leading to this error.

3

That’s what I’m trying to figure out – as they are produced by mergewith itself.

As you can see from the example, the correct result would be:

Dict(
    "B" => 1g,
    K("C") => 1g,
    K("A") => 2
)

A straightforward alternative to mergewith gets it right:

mymergewith(f, dicts...) =
    Dict(k => f((d[k] for d in dicts if haskey(d, k))...)
         for k in union(keys.(dicts)...))

The same is the case in my more elaborate code. The component Dicts are fine, and the units (or lack thereof) match – but the type detection and promotion mechanisms used in mergewith mess up the addition.

(That’s what I tried to express by 3.92… actually having the correct unit; i.e., it has the correct unit in the argument to mergewith, and yet mergewith tries to perform an addition with the unit-stripped version of the number, for some reason.)

I could try to produce an example triggering the second error as well, but the fact that my posted example doesn’t work seems to be enough to illustrate the issue (or at least an issue :slight_smile:)

4

FWIW for type unstable stuff you are probably better off with DynamicQuantities.jl. That seems to work okay here:

julia> using DynamicQuantities: g

julia> struct K
           s
       end

julia> mergewith(+,
           Dict(
               K("A") => 1,
               "B" => 1g
           ),
           Dict(
               K("A") => 1,
               K("C") => 2g,  # I added this one too, for demonstration
           ),
           Dict(
               K("C") => 1g
           )
       )
Dict{Any, Number} with 3 entries:
  "B"    => 0.001 kg
  K("C") => 0.003 kg
  K("A") => 2
5

Interesting. Thanks!

Yeah, the problem is that I’ve essentially got sparse vectors in a vector space with an unknown set of named dimensions, so a type-stable setup with structs or (named) tuples wouldn’t be that easy.

An issue is that I’m using UnitJuMP.jl, though I guess I could do any conversions manually.

Or maybe DynamicQuantities automatically converts everything to the same units automatically – as with the kg in your example – so it’s just a matter of getting the values? (I do have some output stuff, where I’d like to use g or μg etc., though. :thinking:.)

6

For displaying as g or μg you can do

using DynamicQuantities.SymbolicUnits: g, μg

or use the 1.0us"g" macro (instead of the usual 1.0u"g"). Just note that you will probably want to do operations in regular dimensions rather than symbolic ones – use uexpand and uconvert to go back and forth.

There are built-in converters to Unitful btw: GitHub - SymbolicML/DynamicQuantities.jl: Fast physical quantities, by treating units as values rather than types

7

Indeed, this appears to be a bug in mergewith in Julia, would be nice to create an issue. Unitful performs correctly here.
Another invocation that should work but fails, and is probably related: mergewith(tuple, Dict("A"=>1), Dict("A"=>2)).

8

Seems indeed related to the implicit type conversions, i.e., it works if you convert all arguments to a common type:

mergewith(+,
          Dict{Any,Number}(
               K("A") => 1,
               "B" => u"1g"
          ),
          Dict{Any,Number}(
               K("A") => 1,
          ),
          Dict{Any,Number}(
               K("C") => u"1g"
          ))

Same for the tuple example:

mergewith(tuple, Dict{String,Union{Tuple,Int64}}("A"=>1), Dict("A"=>2))
# Somehow, here it's enough to convert the first argument?
1 Like
9

It can be enough to change the type of the first Dict only:

mergewith(+,
          Dict{Any,Any}(
               K("A") => 1,
               "B" => u"1g"
          ),
          Dict(
               K("A") => 1,
          ),
          Dict(
               K("C") => u"1g"
          ))

But curiously, changing the first Dict to Dict{Any, Number} causes an error. Tracking this error shows it results from:

julia> promote_rule(Quantity{Number},Number)
Quantity{Number}

which might be a bug as Quantity{Number} is a subtype of Number. The error is in:

Base.promote_rule(::Type{Quantity{S}}, ::Type{T}) where {S,T <: Number} =
    Quantity{promote_type(S,T)}

which allows T to be Number. Perhaps a specialized promotion rule for T=Number should be added for a fix.

10

Just for posterity, the earlier answer I give didn’t have complete type stability. If you do want that, you can get it like this:

julia> using DynamicQuantities

julia> struct K
           s
       end

julia> mergewith(+,
           Dict(
               K("A") => us"1",  # dimensionless 1
               "B" => 1us"g",
           ),
           Dict(
               K("A") => us"1",
           ),
           Dict(
               K("C") => 1us"g",
           )
       )
Dict{Any, Quantity{Float64, SymbolicDimensions{DynamicQuantities.FixedRational{Int32, 25200}}}} with 3 entries:
  "B"    => 1.0 g
  K("C") => 1.0 g
  K("A") => 2.0

You can register new units with @register_unit. It will still be type stable. (Symbolic units are stored as a sparse vector of exponents, with respect to a vector of symbols)

1 Like