Add data to a DataFrame

Specific Domains Data
1

Dear All,

Description of the problem: How can I add data to individual elements of a dataframe generated using a template? For example, based on the rowTemplate (see the example code) I create a Dataframe that I want to fill with values (randDict) at specific points in the DataFrame. The problem is when I used a comprehension + push! the data is copied “in block” i.e., for each row I have all the data. The objective is to have one element per row

Toy Problem

using DataFrames
using Pipe

rowTemplate=DataFrame("a"=>Dict("aa"=>[],"ab"=>[]),"b"=>Dict("ba"=>[],"bb"=>[]),"c"=>Dict("ca"=>[],"cb"=>[]))

dataFrame=@pipe fill(rowTemplate,100)|>vcat(_...)

randVec=rand(1:100,100)
dictRand=[Dict("dic"=>randVecTmp) for randVecTmp in randVec]

[push!(dataFrame[ind,"b"]["ba"],dictRand[ind]) for ind=1:100]

# the objective is to have only one element in the vector and not 100 elements
# i.e.,  dataFrame[1,"b"]["ba"]=dictRand[1];...;dataFrame[100,"b"]["ba"]=dictRand[100];
# and not  dataFrame[1,"b"]["ba"]=dictRand; ....; dataFrame[100,"b"]["ba"]=dictRand;
dataFrame[1,"b"]["ba"]==dictRand # is true
2

It is not fully clear what you want. Could you please show your code in code-block and make it runnable + show what you want instead what you get?

Thank you!

3

Thanks for the advice! I have already updated the post according to your suggestion

4

Instead of:

Try:

dataFrame=vcat([
  select(rowTemplate, All() .=> deepcopy; renamecols=false)
  for _ in 1:100]...)

Which is closer to the intention (IIUC).
And similarly, instead of:

Try:

for ind in 1:100
    push!(dataFrame[ind,"b"]["ba"],dictRand[ind])
end

Which doesn’t create a vector through comprehension syntax.

I have to admit, the toy problem testing deepcopying of structures in DataFrame eluded me for a minute. And it is right that a specific deepcopy method for DataFrames might be a good addition for DataFrames, similar to the copy method. Maybe an issue can be opened.

1 Like
5

OK. So this should solve your problem I think:

dataFrame = reduce(vcat, deepcopy.(fill(rowTemplate, 100)))
6

Thank you!
Adding a deepcopy solves the problem in the toy and the real case (which is substantially more complex ).

7

could this task not be entrusted to kwarg copycols?
copycols= deeptrue :smiley: