Multi layer Dict merge

I was trying to merge multi-layered Dict which is…

x = [ 
 Dict("d1"=>"A1")
 Dict("d1b"=>Dict("d2a"=>"B1"))
 Dict("d1b"=>Dict("d2b"=>"C1"))]


julia>merge(x...) 
Dict{String,Any} with 2 entries:
  "d1"  => "A1"
  "d1b" => Dict("d2b"=>"C1")

the second layer of a dictionary is being overwritten, so I could do this


julia>merge(merge, x...)
Dict{String,Any} with 2 entries:
  "d1"  => "A1"
  "d1b" => Dict("d2b"=>"C1","d2a"=>"B1")

And I wanted to merge the third and fourth layer of a dictionary as well.
but merge(merge, x...) wouldn’t work here.

x = [ Dict("d1"=>"A1")
        Dict("d1b"=>Dict("d2a"=>"B1"))
         Dict("d1b"=>Dict("d2b"=>"C1"))
          Dict("d1b"=>Dict("d2c"=>Dict("d3a"=>"D1")))
           Dict("d1b"=>Dict("d2c"=>Dict("d3b"=>"E1")))]

julia> merge(merge, x...)
Dict{String,Any} with 2 entries:
  "d1"  => "A1"
  "d1b" => Dict{String,Any}("d2b"=>"C1","d2a"=>"B1","d2c"=>Dict("d3b"=>"E1"))

third layer "d2c"=>Dict("d3a"=>"D1") are being overwritten here

It would be nice if I could accomplish this while preserving beautiful julia syntax :slight_smile:

How about

recursive_merge(x::AbstractDict...) = merge(recursive_merge, x...)
julia> recursive_merge(x...) # with your last x
Dict{String,Any} with 2 entries:
  "d1"  => "A1"
  "d1b" => Dict{String,Any}("d2b"=>"C1","d2a"=>"B1","d2c"=>Dict("d3b"=>"E1","d3a"=>"D1"))
8 Likes

I’m on mobile, so I’m not going to write it out, but I think your best bet might be some sort of recursive function with a loop or two. You can loop through keys with for k in keys (mydict), and check if a dictionary has a key with haskey(mydict, key).

Edit: oops, someone beat me to it! Better solution too :slightly_smiling_face:

wow, it works great!
Didn’t knew varags function unwarps dictionary and It can be used such way :laughing:

Thank you!

No, that’s not what’s going on; the varargs is just because the merge methods in Base also support merging multiple dictionaries at once, and in fact that’s the functionality you’re using by splatting the 5-element x vector. It may be more instructive to compare two-argument versions of recursive_merge and the merge(merge, x...) you already came up with:

recursive_merge(x1::AbstractDict, x2::AbstractDict) = merge(recursive_merge, x1, x2)
not_so_recursive_merge(x1::AbstractDict, x2::AbstractDict) = merge(merge, x1, x2)

I’ll let you think about the difference for a bit; as a hint, to untangle things and simplify the thought process it may be helpful to think of merge(x1, x2) and merge(combine, x1, x2) as a completely different functions (e.g., with the latter called merge_with_combine(combine, x1, x2).

1 Like

In case that there is duplicate key in one of the dicts, the above example of recursive_merge fails with

ERROR: MethodError: no method matching recursive_merge(::String, ::String)

If you add a second less specific method, you can even merge dictionaries with identical keys.

#recursively merge kw-dicts
recursive_merge(x::AbstractDict...) = merge(recursive_merge, x...)
# if values are not AbstractDicts, take the last definition (as does merge)
recursive_merge(x...) = x[end]

Here I chose to keep the last entry, because it follows the convention that has been chosen for merge.

3 Likes

For anyone who found this on Google while looking for a recursive merge of JSON-like data structures, here’s a third method which allows this to work with vectors (assume them to be unordered sets)

recursive_merge(x::AbstractDict...) = merge(recursive_merge, x...)
recursive_merge(x::AbstractVector...) = cat(x...; dims=1)
recursive_merge(x...) = x[end]

Example:

7 Likes