Another UndefRefError: access to undefined reference

Specific Domains Data
1

Is there any way to check a Dataframe to see why this error would be returned when running a function on a DF? The error message is too vague. I got this error when running:

unique!(df,[col1,col2,col3])

I’ve only have only received this error on 1 dataset and I clean out missing values beforehand and don’t use undef.

StackTrace

ERROR: UndefRefError: access to undefined reference

Stacktrace:

[1] getindex

   @ .\essentials.jl:917 [inlined]

[2] _broadcast_getindex

   @ .\broadcast.jl:644 [inlined]

[3] _getindex

   @ .\broadcast.jl:674 [inlined]

[4] _broadcast_getindex

   @ .\broadcast.jl:650 [inlined]

[5] getindex

   @ .\broadcast.jl:610 [inlined]

[6] copyto_widen!(res::Vector{…}, bc::Base.Broadcast.Broadcasted{…}, pos::Int64, col::Int64)

   @ DataFrames C:\Users\programmer8\.julia\packages\DataFrames\kcA9R\src\other\broadcasting.jl:27

[7] copy(bc::Base.Broadcast.Broadcasted{DataFrames.DataFrameStyle, Tuple{…}, typeof(coalesce), Tuple{…}})

   @ DataFrames C:\Users\programmer8\.julia\packages\DataFrames\kcA9R\src\other\broadcasting.jl:77

[8] materialize(bc::Base.Broadcast.Broadcasted{DataFrames.DataFrameStyle, Nothing, typeof(coalesce), Tuple{…}})

   @ Base.Broadcast .\broadcast.jl:872

[9] top-level scope

   @ c:\data\process.jl:111

Some type information was truncated. Use `show(err)` to see complete types.
2

Nobody will be able to say anything without a dataset to reproduce this with, or at least a stacktrace :slight_smile:

3

I’ll recreate the stack trace. I cannot share the dataset and am unsure on how to create a sample dataset that would reproduce the issue.

4

I’d hazard a guess that there is an unassigned array element somewhere; that’s not the same as being assigned to a missing value. Check:

((col, name) -> (name, isbitstype(eltype(col)), all(isassigned.(Ref(col), eachindex(col)))) ).(eachcol(df), names(df))

If I guessed right, it’ll show for each column whether its elements can have a reference and if so, whether they are all assigned. If you see 0, 0 then that’s a column that can throw the error. If that’s the case, you’ll have to reevaluate how you’re instantiating the DataFrame, and you’ll also have to consider the columns that can’t have references because those elements just silently hold garbage values if not assigned.

5 
ERROR: MethodError: no method matching keys(::DataFrame)
6

You’ll need to explain the context, paste exact code, and paste the full stack trace. I could imagine it happening with eachindex(::DataFrame) but that’s not happening. evidently happening but shouldn’t.

7

That’s from running the line of your code with the dataframe name modified to fit my needs.

Stacktrace:
 [1] eachindex(itrs::DataFrame)
   @ Base .\abstractarray.jl:318
 [2] (::var"#21#22")(col::Vector{String}, name::String)
   @ Main c:\data\process.jl:111
 [3] _broadcast_getindex_evalf
   @ .\broadcast.jl:678 [inlined]
 [4] _broadcast_getindex
   @ .\broadcast.jl:651 [inlined]
 [5] getindex
   @ .\broadcast.jl:610 [inlined]
 [6] copy
   @ .\broadcast.jl:911 [inlined]
 [7] materialize(bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{…}, Nothing, var"#21#22", Tuple{…}})
   @ Base.Broadcast .\broadcast.jl:872
 [8] top-level scope
   @ c:\data\process.jl:111
Some type information was truncated. Use `show(err)` to see complete types.
8

Yeah that could be a problem, hence the recommendations to paste the code and stacktrace. If you can’t reproduce the DataFrame object directly, could you at least show typeof(df), eltype.(eachcol(df)) as well?

9

show(eltype.(eachcol(allNcoa))) returns a vector of the 74 cols as String types

10

Alright, any one of those could have an undef element. And the line that caused the eachindex(itrs::DataFrame) error?

11

Trying to find something that would show the undef.
Both of these also give the undef error:
VSCodeServer.vscodedisplay(df)

println(ismissing.(df))

12

this is not how you do it. You want to use isassigned(col, idx) to check if a given index in a column is assigned or not, before trying to access it (which is what ismissing.(col) would do)

13

Is that what @Benny mentioned above???