# Dataframe data manipulation

**URL:** <https://discourse.julialang.org/t/dataframe-data-manipulation/136950>\
**Category:** New to Julia\
**Tags:** dataframes, data\_structures\
**Created:** [May 1, 2026, 3:40pm UTC](https://discourse.julialang.org/t/dataframe-data-manipulation/136950 "2026-05-01T15:40:40Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![SergeantMike67](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sergeantmike67/32/25103_2.png) [@SergeantMike67](https://discourse.julialang.org/u/SergeantMike67)\
**Post date:** [May 1, 2026, 3:40pm UTC](https://discourse.julialang.org/t/dataframe-data-manipulation/136950/1 "2026-05-01T15:40:40Z")

</div>

I have a data frame that is formatted like the example below.

```julia-auto
 Row │ Pairwise p_adj boo Factor IntBoo 
     │ String Float64 Bool String Int64
─────┼────────────────────────────────────────────────────────────────────────────
   1 │ Backwoods Blueberries Coville-12… 0.999599 false Gall Weight 0
   2 │ Backwoods Blueberries Coville-12… 1.0 false Gall Length 0
   3 │ Backwoods Blueberries Coville-12… 0.804792 false Gall Width 0
   4 │ Backwoods Blueberries Coville-12… 2.63132e-8 true BodySize 1
   5 │ Backwoods Blueberries Coville-12… 0.0822928 false Egg Count 0
   6 │ Backwoods Blueberries Coville-12… 3.66398e-11 true Egg Volume 1
   7 │ Backwoods Blueberries Coville-12… 0.993763 false Sex Ratio 0
   8 │ Backwoods Blueberries Coville-12… 4.27341e-6 true Parasitoid 1
   9 │ Backwoods Blueberries Jersey-128… 0.996053 false Gall Weight 0
  10 │ Backwoods Blueberries Jersey-128… 0.583841 false Gall Length 0

```

What I want is for Julia to count up the number of significant findings (boo=true), how many total pairwise comparisons and a list of the factors that were significant.

Example;

```julia-auto
Pairwise sig_fact tot_fact Factors that are significant
Backwoods Blueberries Coville-128th Northland 3 5 BodySize, Egg volume, Parasitoid

```

I have the first part done but am having a hard time getting the list of names for the significant factors. The intent is to export to an Excel spreadsheet then included in a Word Document

Mike Sergeant

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [May 1, 2026, 5:19pm UTC](https://discourse.julialang.org/t/dataframe-data-manipulation/136950/2 "2026-05-01T17:19:26Z")

</div>

`combine(groupby(df, :Pairwise), [:Factor, :boo] => ((f, b) -> Ref(f[b])) => :Factors)`  
with MWE:

```julia-auto
using DataFrames, Random;
pairWise = vcat(fill("p1", 5), fill("p2", 2));
boo = rand(Bool, 7);
factors = rand(["f1", "f2", "f3"], 7);
df = DataFrame(:Pairwise => pairWise, :boo => boo, :Factor => factors)

```

which gives

```julia-auto
7×3 DataFrame
 Row │ Pairwise boo Factor 
     │ String Bool String 
─────┼─────────────────────────
   1 │ p1 true f1
   2 │ p1 false f2
   3 │ p1 true f1
   4 │ p1 false f3
   5 │ p1 true f3
   6 │ p2 true f3
   7 │ p2 false f2
combine(groupby(df, :Pairwise), [:Factor, :boo] => ((f, b) -> Ref(f[b])) => :Factors)
2×2 DataFrame
 Row │ Pairwise Factors            
     │ String Array…             
─────┼──────────────────────────────
   1 │ p1 ["f1", "f1", "f3"]
   2 │ p2 ["f3"]

```

(I did not enforce that the factors are unique for each pairwise value).

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [May 1, 2026, 6:09pm UTC](https://discourse.julialang.org/t/dataframe-data-manipulation/136950/3 "2026-05-01T18:09:26Z")

</div>

With DataFramesMeta I think this is pretty elegant

```julia-auto
julia> df_collaped = @chain df begin 
           @by :Pairwise begin 
               :num_factors = length(:boo)
               :num_sig_factors = sum(:boo)
               :sig_factors = Ref(:Factor[:boo])
           end
       end

```

---

<div class="post-metadata">

**Author:** ![eteppo](https://avatars.discourse-cdn.com/v4/letter/e/90db22/32.png) [@eteppo](https://discourse.julialang.org/u/eteppo)\
**Post date:** [May 2, 2026, 12:45pm UTC](https://discourse.julialang.org/t/dataframe-data-manipulation/136950/4 "2026-05-02T12:45:43Z")

</div>

Personally it helps me if I define the summary function first and then apply it, e.g.

```julia-auto
join_if(x, c) = join(x[c], ", ")
vec_if(x, c) = [x[c]] # or Ref(x[c])? Both seemed to work. 

```

Then split-apply-combine in some package:

```julia-auto
using DataFrames
combine(
    groupby(df, :Pairwise),
    :Factor => length => :tot_fact,
    :boo => sum => :sig_fact,
    [:Factor, :boo] => join_if => "Factors that are significant"
)

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [June 1, 2026, 7:30pm UTC](https://discourse.julialang.org/t/dataframe-data-manipulation/136950/5 "2026-06-01T19:30:01Z")

</div>

```julia-auto
using XLSX, DataFrames
df = DataFrame(XLSX.readtable("dataframe manipulation.xlsx", "Foglio1";infer_eltypes=true))
df.IntBoo = Bool.(df.IntBoo)
gdf = groupby(df, :Pairwise)
combine(gdf, :IntBoo => sum =>:n_sig_fac,[:IntBoo,:Factor]=>((y,x)->[x[y]])=>:sig_fac, 
:Factor=>(x->length(unique(first.(split.(x)))))=>:tot_fac)

```

```julia-auto

julia> combine(gdf, :IntBoo => sum =>:n_sig_fac,[:IntBoo,:Factor]=>((y,x)->[x[y]])=>:sig_fac, 
       :Factor=>(x->length(unique(first.(split.(x)))))=>:tot_fac)
2×4 DataFrame
 Row │ Pairwise n_sig_fac sig_fac tot_fac 
     │ String Int64 Array… Int64   
─────┼────────────────────────────────────────────────────────────────────────────────────────
   1 │ BackwoodsBlueberriesCoville-12… 3 ["BodySize", "Egg Volume", "Para… 5
   2 │ BackwoodsBlueberriesJersey-128… 0 String[] 1

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [June 1, 2026, 10:33pm UTC](https://discourse.julialang.org/t/dataframe-data-manipulation/136950/6 "2026-06-01T22:33:16Z")

</div>

```julia-auto

julia> df.boo = df.boo .== "true"
10-element BitVector:
 0
 0
 0
 1
 0
 1
 0
 1
 0
 0
julia> tot_fac(x)=length(unique(first.(split.(x))))
tot_fac (generic function with 1 method)
julia> combine(gdf, [:boo,:Factor]=>((y,x)->[[x[y]] sum(y) tot_fac(x)])=>[:sig_fac,:n_sig_fac,:tot_fac])
2×4 DataFrame
 Row │ Pairwise sig_fac n_sig_fac tot_fac 
     │ String Any Any Any     
─────┼────────────────────────────────────────────────────────────────────────────────────────
   1 │ BackwoodsBlueberriesCoville-12… ["BodySize", "Egg Volume", "Para… 3 5
   2 │ BackwoodsBlueberriesJersey-128… String[] 0 1

```

I notice that the type of the 3 columns resulting from the transformation is Any and not Int64 and Array[String].  
I think it would be preferable to try the automatic conversion.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [June 4, 2026, 2:37pm UTC](https://discourse.julialang.org/t/dataframe-data-manipulation/136950/7 "2026-06-04T14:37:08Z")

</div>

> [@rocco\_sprmnt21](#):
>
> I notice that the type of the 3 columns resulting from the transformation is Any and not Int64 and Array[String].  
> I think it would be preferable to try the automatic conversion

It seems like a significant fact to me.  
Has anyone formed an opinion on this?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [June 4, 2026, 3:48pm UTC](https://discourse.julialang.org/t/dataframe-data-manipulation/136950/8 "2026-06-04T15:48:45Z")

</div>

This is not surprising. You collect the new columns in a `Vector{Any}` here: `[[x[y]] sum(y) tot_fac(x)])`. If you had used a named tuple with `AsTable` you would not have had this problem.