I have a vector of dicts where some entries contains dicts which I want to process in a DafaFrame. I want to flatten the dicts inside the dicts to make it easier to work with the data. I cannot find the correct command. DataFrame(a) works if a is a vector of dicts of dicts, but I cannot find out ho…

@stene some great solutions are above, i just wanted to throw out one more. using TidierData a1=Dict("a"=> 1, "b"=> Dict("c"=>1, "d"=>2)) df = DataFrame(a1) @unnest_wider(df, b) 1×3 DataFrame Row │ a c d │ Int64 Int64 Int64 ─────┼───────────────────── 1 │ 1 1 …

julia> flattenpair(p) = isa(p.second, Dict) ? p.second : Dict(p) flattenpair (generic function with 1 method) julia> reduce(merge, [flattenpair(p) for p in a1]) Dict{String, Any} with 3 entries: "c" => 1 "a" => 1 "d" => 2 julia> DataFrame(reduce(merge, [flattenpair(p) for p in a1])) 1×3 Data…

A tweak on the solution might be: julia> flattenpair(p) = Dict(isa(p.second, Dict) ? "$(p.first)." .* keys(p.second) .=> values(p.second) : p) julia> reduce(merge, [flattenpair(p) for p in a1]) Dict{String, Any} with 3 entries: "b.c" => 1 "a" => 1 "b.d" => 2 julia> DataFrame(reduce(merge,…

@unnest_wider is great, thanks for sharing it! Albeit it will not work on some JSON data, since keys might be missing. On a more complicated case I got: KeyError: key “xxxx” not found. Will report a separate bug, since the intention of the feature is great. A ready made feature was what I was looki…

transform(df, :b => AsTable) is another solution built into DataFrames. It will not work for JSON data where keys are missing.

[image] stene: After bugs are fixed @unnest_wider is definitely the solution for flattening dicts or unnesting dicts. Then I think you could uncheck my answer and mark @drizk1 's as “solution”, to make your preferred solution easier to discover.

[image] stene: Albeit it will not work on some JSON data, since keys might be missing. On a more complicated case I got: KeyError: key “xxxx” not found. Will report a separate bug, since the intention of the feature is great. A ready made feature was what I was looking for. Maybe there is so…

A more complicated example is this one and this is a toy example of what you might get after parsing a JSON file. It reproduces the exact same error as using a real world JSON file. a1=Dict(“a”=>1, “b”=>Dict(“c”=>1, “d”=>2)) a2=Dict(“a”=>1, “b”=>Dict(“c”=>1)) a=[a1;a2] df=DataFrame(a) Will not …

hey thanks for catching this. the fix was quite straightforward and is now in a PR. Please let us know if you run into other edge cases using Pkg; Pkg.add(url = "https://github.com/TidierOrg/TidierData.jl", rev = "unnest_wider_bugfix") a1=Dict("a"=>1, "b"=>Dict("c"=>1, "d"=>2)); a2=Dict("a"=>1, "b…

Thanks, I am also posting a toy example in JSON which should work: test = JSON.parse(“{"arr":[{"a":1,"b":2},{"a":2}]}”) df=DataFrame(test) noLongerABug=@unnest_wider(df, arr)

▶ Read nesting JSON and flat nested DataFrame ▶ Some less toy-like outputs Perhaps with some modification these functions could suit your needs.

Flatten dicts of dicts in DataFrame

General Usage

rocco_sprmnt21 February 18, 2025, 9:30am 8

Maybe there is something here that (suitably adapted) could be useful for you. Can you provide some minimal but exhaustive examples of the cases that interest you?

Topic		Replies	Views
Flattening YFinance.jl JSON result into a DataFrame New to Julia question	22	1421	April 11, 2023
Side effects (or intended effects) in TidierData? Data tidier	11	388	October 31, 2025
Dash datatable using a nested dict anyone done it? New to Julia dataframes , dash	7	1368	June 15, 2022
Unnesting columns of a data frame containing arrays General Usage question , dataframes , data_structures	1	533	December 9, 2020
Nested access of DataFrame New to Julia dataframes	8	719	June 25, 2021

Flatten dicts of dicts in DataFrame

Related topics