# Date.value doesn't work with missing values

**URL:** <https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887>\
**Category:** New to Julia\
**Created:** [September 15, 2023, 11:05am UTC](https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887 "2023-09-15T11:05:45Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ddrake](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@ddrake](https://discourse.julialang.org/u/ddrake)\
**Post date:** [September 15, 2023, 11:05am UTC](https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887/1 "2023-09-15T11:05:46Z")

</div>

I have a dataframe with some Dates in it, and I want a new column or vector with the number of days between those dates. So this works:

```
vcat([missing], [y - x for (x, y) in zip(df[:,:date][1:end], df[:,:date][2:end])])

```

The first value is `missing` because there’s no previous date. I can add that to my dataframe and all seems well.

Now I want to do arithmetic with those. It seems I need to use `Date.value` on those to turn them into integers/numbers, I was expecting that the `missing` value would be propagated and remain `missing`, but I get:

```
MethodError: no method matching value(::Missing)
etc etc etc

```

For now, I’m just wrapping this in a simple expression:

```
if ismissing(d) missing else Dates.value(d)

```

but it seems like `Date.value` should handle that. Is this just something that hasn’t been implemented, or is there something about the semantics of `missing` and such that I am, well, missing? Is there a more idiomatic way to handle this?

---

<div class="post-metadata">

**Author:** ![ddrake](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@ddrake](https://discourse.julialang.org/u/ddrake)\
**Post date:** [September 15, 2023, 11:12am UTC](https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887/2 "2023-09-15T11:12:34Z")

</div>

BTW, as a bit of context: I’m experimenting with using a Pluto notebook and dataframes as a replacement for spreadsheets. I track things like our monthly natural gas and electricity usage in a spreadsheet, and I’d like to see if a Pluto notebook might work better.

I’m populating my data with DataFrame and CSV and it’s great – I use a string with dates like “2023-09-15” in it, and getting my date values with:

```
file = CSV.File(IOBuffer(csv))
df = DataFrame(file)

```

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [September 15, 2023, 11:20am UTC](https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887/3 "2023-09-15T11:20:16Z")

</div>

This is standard behaviour for most functions and what the `passmissing` function wrapper is for, i.e. you want

```julia
julia> passmissing(Dates.value).([now(), missing])
2-element Vector{Union{Missing, Int64}}:
 63830463604704
               missing

```

---

<div class="post-metadata">

**Author:** ![ddrake](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@ddrake](https://discourse.julialang.org/u/ddrake)\
**Post date:** [September 16, 2023, 6:58pm UTC](https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887/4 "2023-09-16T18:58:57Z")

</div>

Excellent! Thanks. I was sure there was something less awkward than my if-else expression.

Am I right that the syntax here is a typical way of applying a function to every element of a vector/array (I’m not clear on the distinction in Julia): that is, you have `myfunc` that takes a single number, and I want to map that over `[2,3,5,7]`, I would do

```
myfunc.([2,3,5,7])

```

? I know I can use a list comprehension, and there’s the actual `map`, but it seems the above is common.

---

<div class="post-metadata">

**Author:** ![maxkapur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maxkapur/32/21208_2.png) [@maxkapur](https://discourse.julialang.org/u/maxkapur)\
**Post date:** [September 16, 2023, 9:30pm UTC](https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887/5 "2023-09-16T21:30:50Z")

</div>

Yes, see e.g. [More Dots: Syntactic Loop Fusion in Julia](https://julialang.org/blog/2017/01/moredots/)

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [September 16, 2023, 10:13pm UTC](https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887/6 "2023-09-16T22:13:54Z")

</div>

> [@ddrake](#):
>
> Now I want to do arithmetic with those.

Can’t the arithmetic be done on the Date objects? For example,

```julia
julia> Date(2022,1,1)-Date(2021,1,1)
365 days

```

works.

As for,

> [@ddrake](#):
>
> The first value is `missing` because there’s no previous date

wouldn’t you want to drop the first record altogether in this case?

Thirdly,

```julia
[y - x for (x, y) in zip(df[:,:date][1:end], df[:,:date][2:end])]

```

can also be obtained using `diff(df.date)`.

---

<div class="post-metadata">

**Author:** ![ddrake](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@ddrake](https://discourse.julialang.org/u/ddrake)\
**Post date:** [September 17, 2023, 11:11am UTC](https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887/7 "2023-09-17T11:11:00Z")

</div>

> [@Dan](#):
>
> Can’t the arithmetic be done on the Date objects?

Yes, but the resulting thing doesn’t work with numbers — if I do `Date(2022,1,1)-Date(2021,1,1)`, and then try to divide, it doesn’t work:

```
julia> 123/(Date(2022,1,1)-Date(2021,1,1))
ERROR: MethodError: no method matching /(::Int64, ::Day)

```

> [@Dan](#):
>
> wouldn’t you want to drop the first record altogether in this case?

That would work, but I don’t want that, for two reasons: first, I don’t want to ignore data; if I have some data from my gas bill or whatever, I’d like that in my dataframe. Second, there are other values where this special case of “missing in the first row, and populated otherwise” doesn’t hold, so more generally I need good strategies – such as `passmissing` – to handle missing values.

Thanks for letting me know about `diff`.

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [September 17, 2023, 1:48pm UTC](https://discourse.julialang.org/t/date-value-doesnt-work-with-missing-values/103887/8 "2023-09-17T13:48:59Z")

</div>

> [@ddrake](#):
>
> Yes, but the resulting thing doesn’t work with numbers

Good point. Here is another way around this conundrum, which keeps the semantics and even adds more semantics:

```julia
julia> using Dates

julia> using Unitful

julia> 123 / Quantity(Date(2022,1,1)-Date(2021,1,1))
0.336986301369863 d^-1

```

If 123 above has a unit as well (such as electrical energy or cooking gas mass), then the result can even contain these units.

In case dropping the first record feels like ignoring data, I’m not so sure, as the calculation tries to get rates of consumption, and using the initial row would mean somehow guessing a previous period for its data. This can be done in case of utilities and hoousehold consumption, and can also be done explicitly by adding a guessed previous record (which would become the dropped initial record). This may sound convoluted, but adding `missing` often just means it will need to be specially addressed from then on, until it is eliminated.

But tastes, differ and many approaches are legitimate.