TableOperations.select

New to Julia
1

Hi,

I have been making use of the Arrow.jl library and am currently experiencing difficulty in attempting to obtain a subset of columns from the original (Arrow) table and writing to another, via the TableOperations.select function

Naively, I attempted this:

symbols = tblFile  |> TableOperations.select(columnSymbols) |> Tables.columntable
Arrow.write(arrowLookupTarget,symbols)

where columnSymbols = Vector{Symbol}()

and other variations like:

symbols = tblFile  |> TableOperations.select(p -> for p in [1: length(columnSymbols)]  columnSymbols[p] end) |> Tables.columntable
Arrow.write(arrowLookupTarget,symbols)

since the documentation for TableOperations.select gives the example:

table_subset = ctable |> TableOperations.select(:C, :A) |> Tables.columntable

How can I progress this?

Also, as an aside, when (trying to) reviewing the source code for “select”: https://github.com/JuliaData/TableOperations.jl/blob/b3703bab6ebb6b019bcf8070062520246b9f0f77/src/TableOperations.jl

I found it to be confusing:

# select
struct Select{T, columnaccess, names}
    source::T
end

"""
    Tables.select(source, columns...) => Tables.Select
    source |> Tables.select(columns...) => Tables.Select
Create a lazy wrapper that satisfies the Tables.jl interface and keeps only the columns given by the columns arguments, which can be `String`s, `Symbol`s, or `Integer`s
"""
function select end

select(names::Symbol...) = x->select(x, names...)
select(names::String...) = x->select(x, Base.map(Symbol, names)...)
select(inds::Integer...) = x->select(x, Base.map(Int, inds)...)

function select(x::T, names...) where {T}
    colaccess = Tables.columnaccess(T)
    r = colaccess ? Tables.columns(x) : Tables.rows(x)
    return Select{typeof(r), colaccess, names}(r)
end

which I know to be my lack of knowledge and as someone pursuing Julia with the intention of creating production grade code, using the source as a guide. I point it out purely as my ability to solve my own issue was hindered: I could not fathom where the iteration takes place. How is the struct bound to the function and then how does the piping work?

Regards