Getting a strange error when trying to join list of DataFrames, what could this be?

General Usage
1

Hi how’s it going?

I’m trying to join a list of DataFrames on a unique column, but it won’t even let me set a single one of those DataFrames to a variable and instead gives me this error.

“AbstractDataFrame is not iterable. Use eachrow(df) to get a row iterator or eachcol(df) to get a column iterator”

I’ve never had this issue in the past with Julia and the same code has worked for months.

Thanks

2

The most recent update to DataFrames.jl changed the join API. Did you upgrade DataFrames.jl to 0.21?

If so, this page shows the new joining functions.

3

Can you post a MWE?

1 Like
4 
    cols = []
    for val in list_of_stuff
        greater_than_zero = filter(row -> row[val]>0,d)
        tx_sum = by(greater_than_zero, :identifier, Symbol(val) => sum)
        tx_mean = by(greater_than_zero, :identifier, Symbol(val) => mean)
        local_cols = [tx_sum,tx_mean]
        
        for df in local_cols
            push!(cols,df)
        end
    end
    
    total_df = cols[1]

    for i = 2:length(cols)
        total_df = join(total_df,cols[i],on = :identifier,makeunique=true,kind = :outer)
    end



AbstractDataFrame is not iterable. Use eachrow(df) to get a row iterator or eachcol(df) to get a column iterator

Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] iterate(::DataFrame) at /home/shonbutani/.julia/packages/DataFrames/kwVTY/src/abstractdataframe/iteration.jl:23
 [3] indexed_iterate(::DataFrame, ::Int64) at ./tuple.jl:84
 [4] top-level scope at In[118]:1
5

Note that this is not an MWE. An MWE is copy-pasteable and doesn’t contain objects defined outside the code sample.

Can you confirm that you get an error for the following?

julia> cols = [DataFrame(a = [1, 2, 3], b = rand(3)) for i in 1:10];

julia> let total_df = cols[1]
       for i in 2:length(cols)
           total_df = join(total_df, cols[i], on = :a, kind = :outer, makeunique = true)
       end
       total_df
       end
2 Likes
6

I don’t get an error for that code you sent. Strange. Something to do with the by() method changing the type of DataFrame?

7

No, that is most likely not the issue.

You should try and make a MWE that we can copy and paste that reproduces the error.

Note that I can get the error by doing this:

julia> for i in DataFrame(rand(5,5))
       @show i
       end
ERROR: AbstractDataFrame is not iterable. Use eachrow(df) to get a row iterator or eachcol(df) to get a column iterator

So look for code that does something like that.

1 Like