# Changing many rows to single row julia1.5.3

**URL:** <https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615>\
**Category:** Data\
**Tags:** question\
**Created:** [December 10, 2020, 5:02pm UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615 "2020-12-10T17:02:37Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![orthoeng](https://avatars.discourse-cdn.com/v4/letter/o/977dab/32.png) [@orthoeng](https://discourse.julialang.org/u/orthoeng)\
**Post date:** [December 10, 2020, 5:02pm UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615/1 "2020-12-10T17:02:37Z")

</div>

I have a dataset that is 1miil x 70 but with in the rows many columns have duplicate entries. Likewise there are many rows that are unique columns also.

Example  
`| ID | age | location | street | number |`  
`| 1 | 2 | A | E | 23 |`  
`| 1 | 2 | B | G | 23 |`  
` | 2 | 2 | C | G | 34 |`  
` | 2 | 2 | D | G | 34 |`

would like it to look like this.

` | ID | age | location | street | number |`  
` | 1 | 2 | (A ,B ) | (E, G) | (23, 23) |`  
` | 2 | 2 | (C, D) | (G, G) | (34, 34) |`

I did a small code to try and understand the problem. But I’m not sure how to do it in a large scale problem. When I don’t know all the columns that will have duplicate values.

df = DataFrame(x = [1, 1, 1, 1, 2, 2, 2, 2], y = [“a”, “b”, “c”, “d”, “a”, “b”, “c”, “d”])  
gdf = groupby(df, :x)  
change\_col = gdf[1][:,2]  
change\_col = reshape(gdf[1][:,2], 1, : )  
rowuniq = unique(gdf[1][:,1])  
test = [rowuniq, change\_col]

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [December 12, 2020, 1:22pm UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615/2 "2020-12-12T13:22:24Z")

</div>

I’m not sure I understand what the expected result is, but the example I illustrate could be a good approximation of what you are looking for or, hopefully, a starting point in the right direction.

 ![immagine](https://global.discourse-cdn.com/julialang/original/3X/4/1/41773f7db6dca8b20d2ed26210da6ab79e7106db.png)

It’s certainly not clear what you mean by referring to a situation where you don’t know which columns have repeating data.  
I believe that you have to choose, on the basis of the objective you have, the columns on which to group the repeated values.

---

<div class="post-metadata">

**Author:** ![orthoeng](https://avatars.discourse-cdn.com/v4/letter/o/977dab/32.png) [@orthoeng](https://discourse.julialang.org/u/orthoeng)\
**Post date:** [December 12, 2020, 5:27pm UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615/3 "2020-12-12T17:27:31Z")

</div>

Hello rocco, Sorry this is confusing.

What I have is many folder mark by years in each folder I have many excel files that I’m combining. Not all the years have the same type of column structure. I have a dataset that is around about (1miil x 70) + but with in the rows many columns have duplicate entries. Likewise there are many rows that are unique columns also. When I look at the head I see a lot of duplicate entries but each rows is unique so there is some unique columns somewhere in 70 +. So I need some way that I can get duplicates out but keep the unique values. Because I have so many columns it’s hard to figure out where all the unique values will be likewise for the duplicates. I modified the example I see I made some takes last time.

Example  
`| ID | age | location | street | number |`  
`| 1 | 2 | A | E | 23 |`  
`| 1 | 2 | B | G | 23 |`  
`| 2 | 2 | C | G | 34 |`  
`| 2 | 2 | D | G | 34 |`

would like it to look like this.

`| ID | age | location | street | number |`  
`| 1 | 2 | (A ,B ) | (E, G) | (23) |`  
`| 2 | 2 | (C, D) | (G) | (34) |`

---

<div class="post-metadata">

**Author:** ![kmundnic](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@kmundnic](https://discourse.julialang.org/u/kmundnic)\
**Post date:** [December 12, 2020, 5:55pm UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615/4 "2020-12-12T17:55:45Z")

</div>

This is kind of ugly and possibly not fast, but it’ll be a starting point:

```julia
using DataFrames
df = DataFrame(x = [1, 1, 1, 1, 2, 2, 2, 2], y = ["a", "b", "a", "b", "c", "d", "c", "d"])
df2 = DataFrame(x = Int[], y = Tuple[])

groups = groupby(df, :x)
for group in groups
    x = only(unique(group.x))
    y = Tuple(unique(group.y))
    df2 = vcat(df2, DataFrame(x = x, y = y))
end

println(df2)
2×2 DataFrame
 Row │ x y
     │ Int64 Tuple
─────┼───────────────────
   1 │ 1 ("a", "b")
   2 │ 2 ("c", "d")

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 12, 2020, 6:34pm UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615/5 "2020-12-12T18:34:45Z")

</div>

You porobably want the following

```julia
julia> combine(groupby(df, :x), names(df, Not(:x)) .=> Ref ∘ Tuple ∘ unique .=> names(df, Not(:x)))

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [December 12, 2020, 7:01pm UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615/6 "2020-12-12T19:01:30Z")

</div>

`renamecols=false` now keeps the source column names also.

And `Ref` is not needed as `Tuple` is by default left “as is”.

---

<div class="post-metadata">

**Author:** ![orthoeng](https://avatars.discourse-cdn.com/v4/letter/o/977dab/32.png) [@orthoeng](https://discourse.julialang.org/u/orthoeng)\
**Post date:** [December 13, 2020, 3:57am UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615/7 "2020-12-13T03:57:23Z")

</div>

Thank you @rocco_sprmnt21, @kmundnic, @pdeffebach, @bkamins for all the help.

@pdeffebach you Blow my mind what is this part of the code (.=\> Ref ∘ Tuple ∘ unique) also what is this ∘?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [December 13, 2020, 5:52am UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615/8 "2020-12-13T05:52:09Z")

</div>

It’s function composition (types `\circ<tab>`. [Function composition - Wikipedia](https://en.wikipedia.org/wiki/Function_composition)

---

<div class="post-metadata">

**Author:** ![orthoeng](https://avatars.discourse-cdn.com/v4/letter/o/977dab/32.png) [@orthoeng](https://discourse.julialang.org/u/orthoeng)\
**Post date:** [December 13, 2020, 11:51pm UTC](https://discourse.julialang.org/t/changing-many-rows-to-single-row-julia1-5-3/51615/9 "2020-12-13T23:51:12Z")

</div>

@Oscar_Smith Thank you.