# Transpose dictionary

**URL:** <https://discourse.julialang.org/t/transpose-dictionary/82148>\
**Category:** General Usage\
**Tags:** question, performance, array, dictionary\
**Created:** [June 2, 2022, 7:18pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148 "2022-06-02T19:18:31Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 2, 2022, 7:18pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/1 "2022-06-02T19:18:32Z")

</div>

Hello,

I need to transpose a dictionary, but my idea is to use the temporary dictionary X and copy the elements in a transposed manner. Exist something for the critical performance section where the transposition of a dictionary is needed. Here is an example of actual code:

```julia
using DataFrames
using Dates

df_regional_daily = DataFrame(
        DateTime = Date[],
        MonthSin = Float32[],
        MonthCos = Float32[],
        DaySin = Float32[],
        DayCos = Float32[],
        Name = String[],
        Latitude = Float32[],
        Longitude = Float32[]
)
for j in 1:16
    df_regional_daily[!, "Value$(j)"] = Float32[]
end

```

type or paste code here

```julia
X = Dict{String, Array{Float32}}() # temporary
for f in 1:length(data_regional["features"])
    irradiance = data_regional["features"][f]["properties"]["parameter"]["ALLSKY_SFC_SW_DWN"]
    for (t, value) in irradiance
        if haskey(X, t)
            push!(X[t], value) # copying
        else
            X[t] = [value] # creating
        end
    end
end

for (t, value) in X
    t = Date(t, "yyyymmdd")
    push!(df_regional_daily, [
        t,
        sinpi(month(t) / MONTH_PERIOD * 2),
        cospi(month(t) / MONTH_PERIOD * 2),
        sinpi(dayofyear(t) / DAY_PERIOD * 2),
        cospi(dayofyear(t) / DAY_PERIOD * 2),
        location_name,
        location[1],
        location[2],
        value...
    ])
end

```

In the worst case, this code is not memory and computation effective and works with O(N^2)…  
Thanks.

---

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 3, 2022, 4:44am UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/3 "2022-06-03T04:44:26Z")

</div>

Thanks, the `i` index was my mistake. But it doesn’t affect the logic of the code.

---

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 3, 2022, 4:49am UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/4 "2022-06-03T04:49:23Z")

</div>

The full code is [here](https://github.com/markub3327/NASA-POWER-BOT).

---

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 3, 2022, 8:53am UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/6 "2022-06-03T08:53:41Z")

</div>

Now, I’m using JSON.jl and it gets me a nested dictionary that I try to process manually to create a filtered DataFrame. Now, I’m looking for a better solution without using not standard libraries (JSON2.jl or LazyJSON.jl).

Where is the relation between the other thread? Sorry, I did not see it.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 3, 2022, 11:44am UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/7 "2022-06-03T11:44:12Z")

</div>

> [@markub3327](#):
>
> I need to transpose a dictionary

What does this mean? For a matrix, transposition swaps `A[i,j]` and `A[j,i]`. For a dictionary, it means…?

Do you mean swapping keys with values, i.e. going from `d[k] == v` to `d′[v] == k`, i.e. inverting the dictionary?

Or do you mean swapping `d[i][j]` with `d[j][i]` in a multilevel dictionary, similar to [this Python package](https://pypi.org/project/transpose-dict/)?

---

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 3, 2022, 12:03pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/8 "2022-06-03T12:03:36Z")

</div>

Yes, like the Python package. But on the other hand `typeof(data_regional["features"])` is `Vector{Any}`. I mean swapping data\_regional[“features”][f][“properties”][“parameter”][“ALLSKY\_SFC\_SW\_DWN”][t] to d’[t][f], where d’{String, Vector{Any}}().

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 3, 2022, 12:09pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/9 "2022-06-03T12:09:23Z")

</div>

> [@markub3327](#):
>
> data\_regional[“features”][f][“properties”][“parameter”][“ALLSKY\_SFC\_SW\_DWN”][t] to d’[t, f],

Sounds like just:

```julia
d′ = Dict()
features = data_regional["features"]
for f in keys(features)
    tdict = features[f]["properties"]["parameter"]["ALLSKY_SFC_SW_DWN"]
    for t in keys(tdict)
        d′[t, f] = tdict[t]
    end
end

```

---

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 3, 2022, 12:26pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/10 "2022-06-03T12:26:58Z")

</div>

Instead of `for f in keys(features)` use `for f in 1:length(features)`, because `features` is type Vector{Any} (vector of dictionaries).

[“ALLSKY\_SFC\_SW\_DWN”] is Dict{String, Float32}.  
String (key) - is a date in format “yyyy-mm-dd”.  
Float32 - is the final value of this nested dictionary.

And instead of:

```julia
for t in keys(tdict)
        d′[t, f] = tdict[t]
end

```

I have:

```julia
for (t, value) in tdict
    if haskey(d′, t)
        push!(d′[t], value) # copying
    else
        d′[t] = [value] # creating Vector{Float32}
    end
end

```

This example uses `d′ = Dict()`, it’s not memory efficient.  
This example uses 2 `for` loops and copying of every item in `dict`, it’s not efficient too.

It is like 2 glasses with different shapes and water. If you have water in the first one and you want to have a different shape you must take it to another glass with a different shape. And this operation is per water molecule … it is so hard…

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 3, 2022, 12:52pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/11 "2022-06-03T12:52:15Z")

</div>

> [@markub3327](#):
>
> This example uses 2 `for` loops and copying of every item in `dict`, it’s not efficient too.

It seems like what you want, by definition, is a copy of every element of `dict`? It’s just a question of what data structure you want to store the copy into.

I used `d′[t, f]`, i.e. a tuple-keyed dictionary, because that’s what [you said you wanted](https://discourse.julialang.org/t/transpose-dictionary/82148/8). Instead, it seems you want `d′[t][f]`, i.e. a dictionary of vectors? (This has the same asymptotic complexity, but of course the constant factors and the memory layout are different.)

“Transposing” to `d′[t][f]`, where `d′[t]` is a vector, is not possible unless your dictionary is non-sparse, i.e. if a given key `t` is defined for every index `f` (whereas `d′[t, f]` is more general). If that’s the case, your solution seems fine, although you could save a bit of time by preallocating the arrays with size `length(features)` rather than doing `push!` one element at a time.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 3, 2022, 12:55pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/12 "2022-06-03T12:55:14Z")

</div>

> [@markub3327](#):
>
> Instead of `for f in keys(features)` use `for f in 1:length(features)`, because `features` is type Vector{Any} (vector of dictionaries).

Note that `keys` works for arrays too.

---

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 3, 2022, 1:01pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/13 "2022-06-03T13:01:18Z")

</div>

Sorry, Yes I want `d′[t][f]`.

For every `f` exists `dict{String, Vector{Float32}}`. I think too, it’s dense.

---

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 3, 2022, 1:05pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/14 "2022-06-03T13:05:58Z")

</div>

`@threads for` macro can help?

---

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 3, 2022, 1:13pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/15 "2022-06-03T13:13:35Z")

</div>

> [@stevengj](#):
>
> `d′[t]` is a vector

It must be a vector, because `features` is a vector, too. Or it may be `dict`, it will be better or without change?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 3, 2022, 1:15pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/16 "2022-06-03T13:15:04Z")

</div>

> [@markub3327](#):
>
> `@threads for` macro can help?

You would have to be careful about race conditions. You definitely can’t use the `push!` solution with threads, but preallocating the `d′[t]` arrays and then writing into them in parallel (parallelizing over the `for f in …` loop) could work, I guess.

You’re unlikely to get much speedup using threads unless your dictionary is truly huge, though.

---

<div class="post-metadata">

**Author:** ![markub3327](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/markub3327/32/36784_2.png) [@markub3327](https://discourse.julialang.org/u/markub3327)\
**Post date:** [June 3, 2022, 1:31pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/17 "2022-06-03T13:31:56Z")

</div>

> [@stevengj](#):
>
> (parallelizing over the `for f in …` loop)

Now, I’m using `@threads for` in another context and can I have `@threads for` in `@threads for`. Nested `@threads for`?

Few millions values.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 3, 2022, 2:51pm UTC](https://discourse.julialang.org/t/transpose-dictionary/82148/18 "2022-06-03T14:51:36Z")

</div>

> [@markub3327](#):
>
> can I have `@threads for` in `@threads for`. Nested `@threads for`?

Yes.