# How to get ranges of valid data in vectors or dataframes?

**URL:** <https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985>\
**Category:** General Usage\
**Tags:** dataframes, vector\
**Created:** [June 18, 2025, 9:49am UTC](https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985 "2025-06-18T09:49:33Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ju\_ska](https://avatars.discourse-cdn.com/v4/letter/j/ecd19e/32.png) [@Ju\_ska](https://discourse.julialang.org/u/Ju_ska)\
**Post date:** [June 18, 2025, 9:49am UTC](https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985/1 "2025-06-18T09:49:33Z")

</div>

Hi,

```julia
using DataFrames

df = DataFrame(col1 = [1, NaN, 3, 4, NaN, 6, 7, 8, 9, 10, NaN, NaN], col2 = [missing, 2, 3, 4, 5, 6, missing, 8, 9, 10, 11, 12])

```

```julia
12×2 DataFrame
 Row │ col1 col2    
     │ Float64 Int64?  
─────┼──────────────────
   1 │ 1.0 missing 
   2 │ NaN 2
   3 │ 3.0 3
   4 │ 4.0 4
   5 │ NaN 5
   6 │ 6.0 6
   7 │ 7.0 missing 
   8 │ 8.0 8
   9 │ 9.0 9
  10 │ 10.0 10
  11 │ NaN 11
  12 │ NaN 12

```

So for either column, I would like to get the maximum ranges of contiguous valid data, e.g. the lines (indexes or actual lines) range 6 to 10 for column 1.  
Or for missings 2 to 6 and 8 to 11 for column 2.  
I started to think about running through the lines etc. but I guess that there’s a more native way to do that.

---

<div class="post-metadata">

**Author:** ![lukketotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lukketotte/32/217106_2.png) [@lukketotte](https://discourse.julialang.org/u/lukketotte)\
**Post date:** [June 18, 2025, 2:26pm UTC](https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985/2 "2025-06-18T14:26:40Z")

</div>

Here would be an example of how I would do it

```julia
function max_valid_run(v)
    max_len = 0
    curr_len = 0
    for x in v
        if x !== missing && !isnan(x)
            curr_len += 1
            max_len = max(max_len, curr_len)
        else
            curr_len = 0
        end
    end
    return max_len
end

mapcols(col -> max_valid_run(col), df)

```

```julia
1×2 DataFrame
 Row │ col1 col2  
     │ Int64 Int64 
─────┼──────────────
   1 │ 5 5

```

Probably is a more efficient way of checking the columns but have to pay some care as ` isnan(missing)` gives `missing` as opposed to `false` so `!isnan(x) && x !== missing` would throw a TypeError

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [June 19, 2025, 12:15am UTC](https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985/3 "2025-06-19T00:15:07Z")

</div>

Same idea, compressed to one liner:

```julia
julia> mapcols(c->maximum(diff(findall(
  (x->ifelse(ismissing(x),true,isnan(x))).(c))))-1, df)
1×2 DataFrame
 Row │ col1 col2  
     │ Int64 Int64 
─────┼──────────────
   1 │ 5 5

```

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [June 19, 2025, 7:38am UTC](https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985/4 "2025-06-19T07:38:16Z")

</div>

The code below outputs vectors of unit ranges as requested:

```julia
using DataFrames

function max_valid_ranges(v)
    ix = findall(in((NaN, missing)), v)
    (1 ∉ ix) && pushfirst!(ix, 0)
    n = length(v)
    (n ∉ ix) && append!(ix, n + 1)
    d = diff(ix)
    iy = findall(==(maximum(d)), d)
    return [(ix[i]+1):(ix[i]+d[i]-1) for i in iy]
end

df = DataFrame(col1 = [1, NaN, 3, 4, NaN, 6, 7, 8, 9, 10, NaN, NaN], col2 = [missing, 2, 3, 4, 5, 6, missing, 8, 9, 10, 11, 12])

max_valid_ranges(df.col1) # [6:10]
max_valid_ranges(df.col2) # [2:6; 8:12]

```

---

<div class="post-metadata">

**Author:** ![Ju\_ska](https://avatars.discourse-cdn.com/v4/letter/j/ecd19e/32.png) [@Ju\_ska](https://discourse.julialang.org/u/Ju_ska)\
**Post date:** [June 23, 2025, 12:55pm UTC](https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985/5 "2025-06-23T12:55:36Z")

</div>

Thanks a lot !  
I’ve never used Julia’s mathematical syntax, I guess that the time has come !

---

<div class="post-metadata">

**Author:** ![digital\_carver](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/digital_carver/32/33818_2.png) [@digital\_carver](https://discourse.julialang.org/u/digital_carver)\
**Post date:** [June 23, 2025, 3:48pm UTC](https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985/6 "2025-06-23T15:48:18Z")

</div>

I assume you’re referring to the `∉`.

`(1 ∉ ix)` reads pretty nice and is my preferred syntax for this by a lot, but note that you can instead write it as `!in(1, ix)` if you prefer or need to avoid non-ASCII in code.

Any Unicode operator or function name in Julia base language will always have an equivalent pure ASCII version of it that works the same. So the mathematical syntax is really nice to have, but always optional.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [June 23, 2025, 4:13pm UTC](https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985/7 "2025-06-23T16:13:54Z")

</div>

to your point,

```julia
const ∈ = in
∉(x, itr) = !∈(x, itr)
∉(itr) = Fix2(∉, itr)

```

these are defined in Base

---

<div class="post-metadata">

**Author:** ![devel-chm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/devel-chm/32/3572_2.png) [@devel-chm](https://discourse.julialang.org/u/devel-chm)\
**Post date:** [June 23, 2025, 6:59pm UTC](https://discourse.julialang.org/t/how-to-get-ranges-of-valid-data-in-vectors-or-dataframes/129985/8 "2025-06-23T18:59:08Z")

</div>

> [@digital\_carver](#):
>
> Any Unicode operator or function name in Julia base language will always have an equivalent pure ASCII version of it that works the same. So the mathematical syntax is really nice to have, but always optional.

Not `xor()`, you must use `⊻` (sigh…)