Concatenating vectors of different length into a dataframe or table

General Usage
1

Hi,
I am trying to concatenate vectors of different length (can we merge vectors of different lengths?) into a dataframe or table but nothing seems to work. I will share my code.


g = loadtable("all_files" ; spacedelim = true, filenamecol = "files")
df1 = DataFrame(g)

Age = DataFrame()
r = []
for i = 2:length(g)
        global Age , r
        if [g.columns.files[i-1]] == [g.columns.files[i]]
                r = append!(r , g.columns.logAge[i-1])
        else
                Age[:,i-1] = r
                r = []
        end
end

Thank you.

2

Could you edit your post to put the code in a code block? That makes the code easier to read. It should look something like this:

using DataFrames

df = DataFrame(a=1:3, b = 4:5)

You can create a code block by placing triple backticks before and after the code, or by using the “Preformatted text” button in the GUI.

Also, it helps if the code you provide is runnable. Unfortunately, we can’t run this line

g = loadtable(“all_files” ; spacedelim = true, filenamecol = “files”)

so that makes it more difficult to help.

3

Here’s some more guidance on questions that will help us to help you. :slightly_smiling_face:

1 Like
4

I put the code in code block.
Thank you for your help.

5

I can’t run your code and I don’t fully understand it because you haven’t posted an MWE that I can easily copy and paste and run everything.

A data frame can’t store columns of different lengths. This might be your problem.

6

Here’s an example of what happens if you try to extend a column of a data frame without extending the other columns by the same amount:

julia> using DataFrames

julia> df = DataFrame(a=1:2, b=3:4)
2Ă—2 DataFrame
│ Row │ a     │ b     │
│     │ Int64 │ Int64 │
├─────┼───────┼───────┤
│ 1   │ 1     │ 3     │
│ 2   │ 2     │ 4     │

julia> append!(df.b, [5, 6])
4-element Array{Int64,1}:
 3
 4
 5
 6

julia> df
Error showing value of type DataFrame:
ERROR: AssertionError: Data frame is corrupt: length of column
:b (4) does not match length of column 1 (2). The column vector
has likely been resized unintentionally (either directly or
because it is shared with another data frame).

Note that the data frame gets corrupted after you extend column b.

7

what is MWE.
yeah I thought so. Is there any other way to store columns of different length into an array.

Thank you.

8

An MWE ns a “minimum working example”. and just means a small set of code that I can copy and paste (without having to download any other files etc.) that results in the error you are experiencing.

You can use a Dictionary


julia> Dict("a" => [1, 2], "b" => [1, 2, 3])
Dict{String,Array{Int64,1}} with 2 entries:
  "b" => [1, 2, 3]
  "a" => [1, 2]
1 Like
9

Doesn’t vcat pad DataFrames with missing? Could that be helpful here?

10

Other tricks with DataFrames:

series = [[4,5], [7,8,9]]
rows = [[1:length(s);] for s in series]
df = flatten(DataFrame(g=["s1","s2"], s=series, r=rows), [:s, :r])
 unstack(df, :g, :s)

For more info: ?flatten and ?unstack

1 Like