Replace all NaN's with zeros in DataFrame

General Usage
22

Or clamp.(df, 0.0, Inf) which is nice and explicit (at least if you know what “clamp” means I guess…)

4 Likes
23

In short - data frame supports broadcasting in full :grinning_face_with_smiling_eyes:.

3 Likes
24

It would sure be nice to have a function replacenan(df,fillwith) that could iterate through a df finding all Floats with NaNs and replacing them with fillwith. I have a slightly complex dataframe:

Screen Shot 2022-10-14 at 5.45.12 PM
Screen Shot 2022-10-14 at 5.45.12 PM3588×288 38.9 KB

And I’ve been trying to figure out how to replace all on the nan’s so I can export as a Parquet file.

I’m sure people have written this… seems like a common task

25

Try

replace.(df, NaN => 0)

which basically applies replace(::Array, NaN => 0) to each cell. Use replace! for in-place version. Nice, huh?

2 Likes
26

@Jollywatt that is a dream! Love it.

1 Like
27

Not perfect but this is how I did it…
transform(addCols, nanCols .=> ByRow(x → replace.(x, r"NaN"=> 0)) .=> nanCols)

28

apparently this work in all cases except NaN … oops

29

@bkamins This will fail in case if df has missings right? how to handle such scenarios?

30

You mean the above? But what do you want to do with missings? Keep them being missing? In this case you need:

ifelse.(isless.(df, 0), 0, df)

(this works in this specific case, in case of other conditions you would need double condition or passmissing)

1 Like
31

I apologize for not mentioning this earlier but below operation will fail in case there are any missing’s in df columns.

df .= ifelse.(isnan.(df), 0, df)

32

I assume you want to keep missing to be still missing

df .= ifelse.(isnan.(coalesce.(df, false)), 0, df)
1 Like
33

Thank You!!