Debugging a script to replace NaN or -999 values with missing values

General Usage
1

I’m trying to write a program to replace all NaN values or -999 values in my matrix B with missing values (write down nothing) before I use Dataframe to put them into CSV files).

Below is my code:

B = [-0 0 1; NaN 0.2 7; 0 NaN -999; NaN 15 NaN];
replace!(x -> (isnan.(x) || x .== -999) ? missing : x, B)

Here is the error message:
MethodError: Cannot convert an object of type Missing to an object of type Float64

Why doesn’t it work? Thanks!

2

B is a Vector{Float64}. Therefore, it can only store Float64 values. If you define B as B = Union{Float64, Missing}[-0 0 1; NaN 0.2 7; 0 NaN -999; NaN 15 NaN];, it would work.

2 Likes
3

Thanks for the reply!

This is a simplified example. In reality, my B matrix is huge and it is read out of another data source. In that case, how do I make the conversion? I tried the below but it did not work:
B = Union{Float64, Missing}(B)

4

You want to use B = Matrix{Union{Float64,Missing}}(B).

1 Like
5

This will make a copy, but that is unavoidable since Matrix{Union{Float64,Missing}} has a different memory layout than Matrix{Float64}

1 Like
6

agreed, but @leon could allocate B first as a Matrix{Union{Float64,Missing}} and then read the data from source into this allocation?

1 Like
7

Many thanks, All. This addressed my issue nicely.

1 Like