# How to eliminate empty columns

**URL:** https://discourse.julialang.org/t/how-to-eliminate-empty-columns/57484
**Category:** Data
**Created:** [March 18, 2021, 4:20pm UTC](https://discourse.julialang.org/t/how-to-eliminate-empty-columns/57484 "2021-03-18T16:20:12Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Laco\_Kovac](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laco_kovac/32/20684_2.png) [@Laco\_Kovac](https://discourse.julialang.org/u/Laco_Kovac)
#### Post date: [March 18, 2021, 4:20pm UTC](https://discourse.julialang.org/t/how-to-eliminate-empty-columns/57484/1 "2021-03-18T16:20:12Z")

</div>

Here is roughly the code which exports data from the table to CSV String

```julia
using LibPQ, DataFrames, CSV

executeResult = execute(conn, "SELECT c1, c2, c3, c4, c5 FROM some_table WHERE some_id = '$someId'")
if (LibPQ.libpq_c.PGRES_TUPLES_OK != LibPQ.status(executeResult))
    throw("Data not found.")
end
bEmptyData = isempty(executeResult)
if !bEmptyData
    frame = DataFrame(executeResult)
    sResult = join(CSV.RowWriter(frame))
end

```

Code works pretty well, but it needs one adjustment.  
In the resulting data set, some of the columns can be completely empty (e.g. result contains 5 rows and in each of them column c4 is NULL/missing). In such case, I am supposed to remove c4 from sResult altogether (not in the header nor comma for it in any row).

What would be the most efficient way to do that?

Thank you.

---

<div class="post-metadata">

### Author: ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)
#### Post date: [March 19, 2021, 5:39am UTC](https://discourse.julialang.org/t/how-to-eliminate-empty-columns/57484/2 "2021-03-19T05:39:24Z")

</div>

I’d probably do something like:

```julia
using TableOperations
io = IOBuffer()
executeResult |> 
  TableOperations.select(:c1, :c2, :c3, :c5) |>
  CSV.write(io)
sResult = String(take!(io))

```

---

<div class="post-metadata">

### Author: ![Laco\_Kovac](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laco_kovac/32/20684_2.png) [@Laco\_Kovac](https://discourse.julialang.org/u/Laco_Kovac)
#### Post date: [March 19, 2021, 8:19am UTC](https://discourse.julialang.org/t/how-to-eliminate-empty-columns/57484/3 "2021-03-19T08:19:09Z")

</div>

Yes, thank you. But I need also the logic to find out which of these 5 columns have all result NULL/missing. c4 I mentioned above was just an example.

---

<div class="post-metadata">

### Author: ![Peter\_Adelman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peter_adelman/32/11907_2.png) [@Peter\_Adelman](https://discourse.julialang.org/u/Peter_Adelman)
#### Post date: [March 19, 2021, 9:44am UTC](https://discourse.julialang.org/t/how-to-eliminate-empty-columns/57484/4 "2021-03-19T09:44:24Z")

</div>

```julia
using LibPQ, DataFrames, CSV

executeResult = execute(conn, "SELECT c1, c2, c3, c4, c5 FROM some_table WHERE some_id = '$someId'")
if (LibPQ.libpq_c.PGRES_TUPLES_OK != LibPQ.status(executeResult))
    throw("Data not found.")
end
bEmptyData = isempty(executeResult)
if !bEmptyData
    frame = DataFrame(executeResult)
    to_remove = String[]
    for name in names(frame)
        coltype = eltype(frame[!, name])
        if coltype != nonmissingtype(coltype) && all(ismissing.(frame[!, name]))
            push!(to_remove,name)
        end
    end
    !isempty(to_remove) && select!(frame, Not(to_remove))
    sResult = join(CSV.RowWriter(frame))
end

```

Here is one option. I believe that LibPQ returns types based on the table schema, so verifying that missing is part of the eltype will only tell you that it’s nullable and thus might not be worth doing in your case, which is why I didn’t filter by that.

Edit: added it because why not

---

<div class="post-metadata">

### Author: ![Laco\_Kovac](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laco_kovac/32/20684_2.png) [@Laco\_Kovac](https://discourse.julialang.org/u/Laco_Kovac)
#### Post date: [March 19, 2021, 12:36pm UTC](https://discourse.julialang.org/t/how-to-eliminate-empty-columns/57484/5 "2021-03-19T12:36:22Z")

</div>

This is it, Peter. Thank you!
