Hi folks, We’re getting a somewhat random segfault on a dataframes getindex operation inside a for-loop. I haven’t been able to extract a MWE from the proprietary code where this is occurring, but here is part of the segfault and some notes on what we’ve tried. Maybe folks will have some recommenda…

did I understand you correctly that on DataFrames 0.17.1 the problem disappears, because earlier you have reported that the bug still persisted? That’s correct, on 0.17.1 I can’t reproduce a segfault. I’ll update my previous comment to make that clearer. Thank you. NOTE: On 0.17.0, setting --che…

Ooops, apparently folks had only been getting the segfault on 0.16.0 and 0.17.0, adding the GC.gc() didn’t actually fix the problem cause I’ve been testing on 0.17.1. FWIW, adding the print/debugging statements for mydataframe appear to prevent the segfault from happening (similar to @show) on 0.17.…

Good to hear that 0.17.1 solves the issue. Thank you. But I would like to make sure that something is not lurking unnoticed. From what you report a change that most likely affected how your code behaves is: https://github.com/JuliaData/DataFrames.jl/commit/8d75d7464bdea76bb394081f66d870b2d9bf1c3b#…

[image] bkamins: This is very strange, as /src/subdataframe/subdataframe.jl:102 in your stack trace is a call like df[:n] not df[1, :n] ( df[1, :n] should call line 107 not 102). @Rory-Finnegan may have switched from x = df[1, :n] to x = first(df[:n]) or possibly x = first(@view df[:n]) whi…

Thank you. This would mean that we are getting a segfault going through different paths in DataFrames.jl (which would suggest some problem with lookup dict corruption). Hopefully this is a bug in DataFrames.jl (as this will be easy to fix). What you say makes me think that it cannot be excluded that…

[image] Rory-Finnegan: NOTE: On 0.17.0, setting --check-bounds=yes cause julia to exit with 'julia-1.0 --check-bounds=yes' terminated by signal SIGSEGV (Address boundary error) This is really weird. In DataFrames.jl we do not do any unsafe operations that directly operate on pointers or simil…

Hmmm, it still segfaults with the overwritten Base.ht_keyindex. On 1.1 it gives me a slightly longer traceback, indicating that the call to isslotempty is erroring. signal (11): Segmentation fault: 11 in expression starting at no file:0 getindex at ./array.jl:729 [inlined] isslotempty at ./dict.jl:…

This is exactly what I have suspected, i.e. the lookup dict gets corrupted somehow (and its slots field is shorter than keys field). Can you try this (sorry for this ping-pong, but I do not know a better way): @noinline function Base.ht_keyindex(h::Dict{K,V}, key) where V where K sz = length(h…

Okay, so the first version successfully threw a KeyError: ERROR: KeyError: key :n not found The second version resulted in a segfault though: signal (11): Segmentation fault: 11 in expression starting at no file:0 getindex at /root/.julia//packages/DataFrames/lyCjP/src/dataframe/dataframe.jl:228 …

I assume that obviously :n is expected to be in the data frame - right? In general this is very strange (especially the last segfault). Also in src/dataframe/dataframe.jl:228 there is index not getindex function defined, so this is strange. Probably having a some shareable reproducing example at …

Segfault on DataFrames `getindex`

General Usage

nalimilan Split this topic March 17, 2020, 6:24pm 26

A post was split to a new topic: Get on GroupedDataFrame

Topic		Replies	Views
Apparently incorrect error when going out of bounds on a dataframe General Usage dataframes	2	387	June 2, 2020
Get on GroupedDataFrame Data	11	1416	March 18, 2020
Cant create dataframe Optimization (Mathematical) jump , dataframes	16	758	September 29, 2020
Release announcements for DataFrames.jl Data announcement , dataframes	190	25709	September 28, 2023
Dropmissing!() on a dataframe causes segfault General Usage	8	514	September 25, 2020

Segfault on DataFrames `getindex`

Related topics