It seems extrema(x,dims=1)
silently drops NaNs - and I cannot find that documented. (As a reference, minimum
does not.) Did I miss something?
Example
x = [1,NaN,3]
println(extrema(x))
println(extrema(x,dims=1))
gives
(NaN, NaN)
[(1.0, 3.0)]
It seems extrema(x,dims=1)
silently drops NaNs - and I cannot find that documented. (As a reference, minimum
does not.) Did I miss something?
Example
x = [1,NaN,3]
println(extrema(x))
println(extrema(x,dims=1))
gives
(NaN, NaN)
[(1.0, 3.0)]
Looks like a bug to me. I went ahead and opened issue #43599.
It seems not yet fixed. Is there an alternative to function extrema()
?
It fails also, if the variable contains elements of type Missing
In my case the following helped:
_z_phase_array = [2.0,2,missing,2,2]
extrema(DataFrames.skipmissing(_z_phase_array))
I works for me on julia v1.9.0-rc2
:
julia> x = [1,NaN,3]
3-element Vector{Float64}:
1.0
NaN
3.0
julia> extrema(x)
(NaN, NaN)
julia> extrema(x,dims=1)
1-element Vector{Tuple{Float64, Float64}}:
(NaN, NaN)
I expect, the output (1,3) and not (NaN,NaN).
Well, the issue #43599 was about to include the NaN
s when checking for extrema.
If you don’t want that, you need to roll your own function, probably.
Or use the workarounds provided in package DataFrames
Yes, you could use those too and then call extrema(..)
on their output.
But this will probably allocate an additional array (besides the one returned from extrema
).
If you want to avoid that too, roll your own version with for loops
Try Skipper
, a lighweight package to efficiently skip arbitrary elements:
julia> using Skipper
julia> x = [1,NaN,3]
julia> extrema(skip(isnan, x))
(1.0, 3.0)
if you don’t want to depend on packages not in Base
x = [1,NaN,3]
extrema(filter(!isnan, x))
x = [1,NaN,missing,3]
extrema(filter(e-> (!ismissing(e)) && (!isnan(e)), x))
as a side note
extrema(filter(e-> (!isnan(e)) & (!ismissing(e)), x)) # works
extrema(filter(e-> (!isnan(e)) && (!ismissing(e)), x)) # doesn't work!!! (due to short circuit)
if you use lazy filter, it doesn’t even allocate
julia> @btime extrema(Iterators.filter(e-> (!isnan(e)) & (!ismissing(e)), $x))
8.300 ns (0 allocations: 0 bytes)
(1.0, 3.0)
julia> @btime extrema(Iterators.filter(e-> (!ismissing(e)) && (!isnan(e)), $x))
8.500 ns (0 allocations: 0 bytes)
(1.0, 3.0)
julia> @btime extrema(Iterators.filter(e-> (!ismissing(e)) & (!isnan(e)), $x))
8.300 ns (0 allocations: 0 bytes)
(1.0, 3.0)