I’ve attempted multiple ways to populate the “missing” values of my DataFrame with the appropriate values,
I created the shell of a DataFrame that looks like this:
ResVert_matrix
1826×372 DataFrame
Row │ decyr AGMT AHID ALAM ALBH ...
│ Float64 Any Any Any Any ...
──────┼────────────────────────────────────────────
1 │ 2001.0 missing missing missing missing ...
2 │ 2001.0 missing missing missing missing ...
.
.
.
And what I’m trying to do is fill the “missing” values with the residual vertical position that I have stored in individual files that correspond to each station name at the appropriate decimal year value in the ResVert_matrix.
But, I’ve either been focusing on the problem for too long and have over-complicated and confused myself, or, I’m just going about this incorrectly in general.
I won’t copy and paste the entire code here, since I think the only issue is the loop I’m trying to implement. But, I will include the variables that I use in the loop to help avoid any confusion, where my ‘for loop’ is implemented just below where this code ends:
# glob to correct file:
glob_matrix = glob("*.tenv3", Dstat_path)
# extract decyr column from ResVert_matrix:
all_years = ResVert_matrix.decyr
# extract header names from ResVert_matrix:
stations = names(ResVert_matrix)
stations = DataFrame(stations[2:end, :], [:station]);
stations = stations.station;
# create (maybe nested?) loop to iterate though files and append to ResVert_matrix
for file in glob_matrix
file_hold = readdlm(file)
file_df = DataFrame(file_hold, [
:station, #1
:date, #2
:decyr, #3
:MJD, #4
:east_m_frac, #5
:vert_m_frac, #6
:east_sigma_m, #7
:vert_sigma_m, #8
:d_pred_east, #9
:d_pred_vert, #10
:res_east, #11
:res_vert, #12
:ol_east, #13
:ol_vert, #14
:YYYY, #15
:mm, #16
:dd, #17
:hh, #18
:day, #19
:week, #20
:day_of_week, #21
:J20000_sec] #22
)
file_decyr = file_df.decyr
file_resvert = file_df.res_vert
file_stat = file_df.station
There are a few versions I’ve tried to go about, but, I keep getting either:
ERROR: MethodError: no method matching keys(::DataFrame)
Closest candidates are:
keys(::Pkg.Registry.RegistryInstance)
@ Pkg ~/julia-1.10.2/share/julia/stdlib/v1.10/Pkg/src/Registry/registry_instance.jl:447
keys(::IndexLinear, ::GroupedDataFrame)
@ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/groupeddataframe/groupeddataframe.jl:574
keys(::Attributes)
@ MakieCore ~/.julia/packages/MakieCore/UAwps/src/attributes.jl:36
...
Stacktrace:
[1] pairs(collection::DataFrame)
@ Base ./abstractdict.jl:172
[2] findfirst(testf::var"#147#148"{String}, A::DataFrame)
@ Base ./array.jl:2199
[3] top-level scope
@ ~/Documents/julia/epochs.jl:84
when I try this method:
for (i, stat) in enumerate(stations)
match_Vidx = findfirst(x -> x[:decyr] == stat, ResVert_matrix)
if match_Vidx !== nothing
set!(ResVert_matrix, :res_vert, match_Vidx, file_resvert[i])
end
end
And I will get this error:
ERROR: MethodError: no method matching setindex!(::DataFrame, ::Vector{Any}, ::typeof(!), ::Vector{Symbol})
Closest candidates are:
setindex!(::DataFrame, ::AbstractMatrix, ::typeof(!), ::AbstractVector)
@ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/dataframe/dataframe.jl:777
setindex!(::DataFrame, ::AbstractVector, ::typeof(!), ::Union{AbstractString, Signed, Symbol, Unsigned})
@ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/dataframe/dataframe.jl:674
setindex!(::DataFrame, ::AbstractDataFrame, ::typeof(!), ::AbstractVector)
@ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/dataframe/dataframe.jl:757
...
Stacktrace:
[1] top-level scope
@ ~/Documents/julia/epochs.jl:85
If I try this method:
for (i, stat) in enumerate(stations)
vIdx = [tt in file_decyr for tt in all_years];
ResVert_matrix[!, [:decyr, :stat, :vIdx]] = file_resvert
end
end
The trouble-shooting that I have attempted as been variations on those two methods. I would convert certain variables, or switch out variables for :
match_Vidx = findfirst(x -> x[:decyr] == stat, ResVert_matrix)
Since that is where the code keeps getting caught up on in that method. It would provide an output if I remove findfirst() so, I think my understanding isn’t quite up to par with how to use it, even though I have read the documentation on it.
I don’t remember exactly what I changed to get this error:
ERROR: MethodError: no method matching getindex(::Float64, ::Vector{Any})
Closest candidates are:
getindex(::Number, ::Integer)
@ Base number.jl:96
getindex(::Number)
@ Base number.jl:95
getindex(::Number, ::Integer...)
@ Base number.jl:101
...
Stacktrace:
[1] (::var"#141#143"{DataFrame, String})(x::Float64)
@ Main ~/Documents/julia/epochs.jl:85
[2] findnext(testf::var"#141#143"{DataFrame, String}, A::Vector{Float64}, start::Int64)
@ Base ./array.jl:2155
[3] findfirst(testf::Function, A::Vector{Float64})
@ Base ./array.jl:2206
[4] top-level scope
@ ~/Documents/julia/epochs.jl:85
But, I tried changing converting stat using convert(Vector{Any}, stat) but, that didn’t change anything.
So, I’m kind of at a loss with how to accomplish this. Any insight and help would be greatly appreciated. I’m sure there is some simple solution and I just haven’t come across it in my attempts to find a solution.
Thank you in advance from a weary grad student… 