# \`DateTime\` arithmetic on \`Microsecond\` (or smaller) scale

**URL:** <https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949>\
**Category:** Internals & Design\
**Tags:** dates, datetime\
**Created:** [August 18, 2023, 12:39pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949 "2023-08-18T12:39:34Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![stephancb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stephancb/32/14243_2.png) [@stephancb](https://discourse.julialang.org/u/stephancb)\
**Post date:** [August 22, 2023, 2:33pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/65 "2023-08-22T14:33:09Z")

</div>

Please also have a look at the recommended [time code formats of the CCSDS](https://public.ccsds.org/Pubs/301x0b4e1.pdf), especially section 3.3 “DAY SEGMENTED TIME CODE (CDS)”:

A high precision binary code would include the millisecond accurate code plus an extra field. The day segmentation allows to gracefully represent time stamps within leap seconds (we still compute things like the nr of seconds between two time stamps assuming each day has 86400 seconds, but it is possible to represent and compare time codes also within leap seconds).

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 22, 2023, 2:48pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/66 "2023-08-22T14:48:24Z")

</div>

> [@stephancb](#):
>
> Please also have a look at the recommended [time code formats of the CCSDS](https://public.ccsds.org/Pubs/301x0b4e1.pdf), especially section 3.3 “DAY SEGMENTED TIME CODE (CDS)”:

That’s a very interesting link - as `Dates` is mostly concerned with human interaction (we already check a lot of things in the constructors), I’d personally prefer a CCS approach. About half of that standard also seems concerned with the interchange of data - a problem `Dates` doesn’t tackle at all, so I think that would be better suited to a community package (which could then convert `DateTime`/a hypothetical `HighPrecisionDateTime` to its internal wireformat for transmission).

---

<div class="post-metadata">

**Author:** ![Daniel\_Gonzalez](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/daniel_gonzalez/32/52412_2.png) [@Daniel\_Gonzalez](https://discourse.julialang.org/u/Daniel_Gonzalez)\
**Post date:** [August 22, 2023, 6:07pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/67 "2023-08-22T18:07:01Z")

</div>

Regarding the original topic, I don’t have a clear opinion on what’s the “correct” behavior for the `DateTime` + `Nanosecond` operation; I think there are some good points on both sides, but it’s hard (for me) to reason about the potential use cases where this situation might occur or about the relative primacy of different abstract principles. I do think, nevertheless, that we should be clear about the intended use cases for the stdlib `DateTime` versus the use cases where a custom format (see TimesDates.jl, AstroTime.jl) might be better suited, since the desired format for each user is different. There is no hypothetical standard DateTime object that is optimal for everyone.

For example, there are several suggestions in this thread to basically double the bit size of the DateTime format so that we gain extra precision and the original concerns are moot. That would not be a problem for most of my use cases, but I worry that we would be increasing the memory footprint and reducing the performance of “data science” use cases, where you might have one or several `DateTime` fields over millions of rows. In fact, I think those use cases where you don’t have high precision requirements, but you want a baseline DateTime type to facilitate interoperability between table/dataframe packages and analysis packages, should be prioritized in the standard library.

As for “high-precision” (or “scientific”) users in the other extreme, namely space, astronomy and geophysics, it seems to me that it’s definitely better to rely on external packages that provide the domain-specific optimized format. This is not just because of number format issues, but also because at those precision regimes there are far more issues than just numeric precision. There are different time scales that advance at different rates (UT, TAI, TDB…) and the conversions between them are not trivial: either they require external data files (like the IERS tables) or a computationally heavy operation that might be truncated to different degrees according to performance/precision preferences. Thus, I think it would not be a good idea to burden the standard library with all those domain-specific choices and concerns, particularly when they might be limiting in other aspects (e.g. ability to represent dates in the past). In that sense, I would be cautious about domain-specific standards as well, unless they work  
well for other contexts.

There might be some “intermediate precision” use cases as well, requiring e.g. precision at the nano-second level, where the current type falls short. Perhaps it’s a system reading a sensor at high frequency, or an oscilloscope, or something similar. Maybe it’s worth to define a stdlib `HighPrecisionDateTime` for that case instead of an external type, I don’t have a strong opinion. It’s still important to highlight here that there might be “externally-sourced” issues if you want to do very precise arithmetic with DateTimes (defined in Universal Time) and real-time periods (which are only “correct” when interpreted as a difference of TAI epochs, not as a difference of UT epochs), or when handling different clock references.

Finally, regarding high-frequency trading, I’m not sure what are their requirements: surely you want accuracy, but also optimal performance? Wouldn’t that require another custom time format where you sacrifice the range of representable DateTimes instead of either?

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [August 22, 2023, 6:12pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/68 "2023-08-22T18:12:51Z")

</div>

> [@Sukera](#):
>
> Would you prefer it to alert a user to the issue, in case they do try it without knowing that it won’t be accurate?

I’m not sure what would be best. For example, it is hard to imagine why these two should give different results:

```julia
dt = DateTime(2012)
dt+ Nanosecond(1e9)
foldl(+,(Nanosecond(1) for _ in 1:1e9),init=dt)

```

So maybe that is in favor of just disallowing it altogether? This is related to the comment about commutativity I suppose.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 22, 2023, 7:20pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/69 "2023-08-22T19:20:09Z")

</div>

> [@Daniel\_Gonzalez](#):
>
> In fact, I think those use cases where you don’t have high precision requirements, but you want a baseline DateTime type to facilitate interoperability between table/dataframe packages and analysis packages, should be prioritized in the standard library.

Dates and times are typically treated as continuous variables in data analysis, so floating point would be more natural here instead of integers.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [August 22, 2023, 7:39pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/70 "2023-08-22T19:39:29Z")

</div>

> [@aplavin](#):
>
> Dates and times are typically treated as continuous variables in data analysis, so floating point would be more natural here instead of integers.

@tim.holy’s point above is well taken, though — for a `DateTime` (i.e. a timestamp relative to some fixed epoch, not a time interval), you want _uniform precision over the whole range of supported dates_ (e.g. if you have nanosecond precision for 2023 dates, you also want nanosecond precision for dates in BCE 753). That inherently argues for _fixed-point_ arithmetic, not floating-point. This is what we have now; to gain more precision we would have to widen the type (e.g. to `Int128`).

Time intervals (`Period` in Julia) are a different matter — for that application, uniform _relative_ precision (i.e. less absolute precision for longer intervals) makes sense, and floating-point arithmetic would be more sensible.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [August 22, 2023, 8:19pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/71 "2023-08-22T20:19:09Z")

</div>

Another option for default DateTime is to have the default be 64 bit seconds and a separate 32 bit nanoseconds, this gets you a memory savings of 4 bytes per value relative to Int128, but doesn’t get you the usually excessive attosecond precision. Nanoseconds are a thing people are using today though. You might want to do something like send 20 packets on the wire and wait for replies and measure the durations and jitter to tens of nanoseconds jitter time or something.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [August 22, 2023, 8:43pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/72 "2023-08-22T20:43:57Z")

</div>

> [@dlakelan](#):
>
> this gets you a memory savings of 4 bytes per value relative to Int128

Not with struct padding (since `Int64` likes to live on 8-byte boundaries):

```julia
julia> struct Foo
          seconds::Int64
          ns::Int32
       end

julia> sizeof(Foo(3,4))
16

julia> sizeof([Foo(1,2), Foo(3,4)])
32

```

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [August 22, 2023, 8:53pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/73 "2023-08-22T20:53:19Z")

</div>

> [@stevengj](#):
>
> Not with struct padding

Gotcha, hadn’t thought of that. So might as well go Int128 and attoseconds.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 23, 2023, 12:12am UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/74 "2023-08-23T00:12:26Z")

</div>

> [@mihalybaci](#):
>
> It would appear that the decision has already been made to end leapseconds.

This does not change the dilemma we have (even if the concept of a leap second had never been invented, for any standard):

> From 2035, leap seconds will be abandoned for 100 years or so and will probably never return.

> [@Eben60](#):
>
> Surely astronomy belongs to the so-called exact sciences, […]
> 
> It is probably possible to estimate the Sun position on a certain date a few millenia ago (but probably impossible to verify the results). But it is probably not possible to tell with the resolution of a minute the duration between the noon January 1st 5000 b.c.

It’s possible to estimate the position of the Sun one millennium or century ago, but the position is not in [Date]Time units, but meters. If you want the position for say a six months ago, then you can subtract the required number of seconds, but if you assume the integer number of seconds (or microseconds possible in the representation), then you will be off by a tiny bit. If you have a leap-second in the meantime, or even if there was none. Over longer stretches of time, if you subtract, even in integer number of nanoseconds, then you may get the wrong day, and the error in days could stretch into multiple days.

Yes, who cares about the exact day back then? But it would change the position of the sun, e.g. whether it’s winter or summer.

I may be exaggerating a bit for effect, but this will be the case in milliseconds, or microseconds, or even nanoseconds, it just changes how far back (or forward) in time you have to go.

And I’m not yet taking into account how the rotation speed of the earth around the sun is not constant. Just that it’s not an integer (or rational) multiple of anything.

---

<div class="post-metadata">

**Author:** ![mihalybaci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mihalybaci/32/13528_2.png) [@mihalybaci](https://discourse.julialang.org/u/mihalybaci)\
**Post date:** [August 23, 2023, 4:02pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/75 "2023-08-23T16:02:20Z")

</div>

> [@Palli](#):
>
> First the easiest part to disagree with “you also want nanosecond precision for dates in BCE 753”.

The first use case I can think of where someone might want better precision is the very specific subfield of archaeoastronomy. Examples: trying to find the sky location of a possible supernovae reported in 500 A.D would require back propagation of both the Earth’s axial precession and proper motions of stars and other night sky objects. The same is true if one wants to find out the reason behind the alignment of Stonehenge or other ancient sites that may have a celestial origin.

I’m not an archaeoastronomer, but it seems to be a good assumption that no one is going to run a simulation backwards 7000 years with attosecond precision. And perhaps people would rather use a pure floating number (Julian dates). But it seems like those should be user choices, not constraints imposed by the package.

Should `Dates` be able to handle every use case? Or should people simply use other packages? My examples seem perfect for `AstroTime`. 🤷

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [August 23, 2023, 5:31pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/77 "2023-08-23T17:31:58Z")

</div>

> [@mihalybaci](#):
>
> I’m not an archaeoastronomer, but it seems to be a good assumption that no one is going to run a simulation backwards 7000 years with attosecond precision.

Well, I can well imagine such a simulation as a PhD topic. Actually a great topic: Put your predictions a couple decades in future (climatology) or a couple of centuries in the past, and you can simulate pretty anything  
/ irony off

To be serious: I am not sure whether orbits of celestial bodies were not chaotic to an extent even under assumption of rigid bodies in the absence of external perturbations. Which they are not, and as for external perturbations: our Sun is not exactly a source of constant radiation.

Sorry for offtopic 🙂

---

<div class="post-metadata">

**Author:** ![mihalybaci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mihalybaci/32/13528_2.png) [@mihalybaci](https://discourse.julialang.org/u/mihalybaci)\
**Post date:** [August 23, 2023, 6:21pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/78 "2023-08-23T18:21:57Z")

</div>

Here is one case I remember specifically from this [“forensic astronomer”](https://www.space.com/27007-astronomy-sleuths-monet-painting.html), who estimated down to the minute when Monet was painting a sunrise in 1835. (I tried to find a link that would work for everyone, apologies if it doesn’t). And things are surely chaotic to a small degree, but it is still possible to estimate the dates/times when, say, the Sun would line up just right at Stonehenge in 4-5000 BC.

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [August 23, 2023, 6:45pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/79 "2023-08-23T18:45:27Z")

</div>

Surely astronomy belongs to the so-called exact sciences, which are called so not because they “admit of absolute precision in their results”, as Wikipedia [maintains](https://en.wikipedia.org/wiki/Exact_sciences), but, by the definition I like most, because they make it possible to estimate the reliability of their results.

It is probably possible to estimate the Sun position on a certain date a few millenia ago (but probably impossible to verify the results). But it is probably not possible to tell with the resolution of a minute the duration between the noon January 1st 5000 b.c., and January 1st, 1970 at 00:00:00 UTC.

As for Monet painting - that was mainly meteorological records anyway.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [August 25, 2023, 8:25pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/80 "2023-08-25T20:25:13Z")

</div>

Split this topic on August 25, 2023, 8:25pm UTC

3 posts were merged into an existing topic: [Target topic for a merge fixup](https://discourse.julialang.org/t/target-topic-for-a-merge-fixup/103205/2)

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [August 23, 2023, 4:56pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/84 "2023-08-23T16:56:33Z")

</div>

Maybe I missed that part of the discussion, because it’s such an obvious suggestion but… why not make DateTime parametric? Then you could keep the current behaviour as is by default, while supporting extra precision with eg. `DateTime{Int128}`. On our side, doubling the memory requirement of DateTime would be a bit of a bummer…

---

<div class="post-metadata">

**Author:** ![Jake](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jake/32/46007_2.png) [@Jake](https://discourse.julialang.org/u/Jake)\
**Post date:** [August 23, 2023, 4:58pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/85 "2023-08-23T16:58:11Z")

</div>

[Sir Isaac Newton](https://www.barnesandnoble.com/w/newtons-revised-history-of-ancient-kingdoms-sir-isaac-newton/1112711452) used some of these techniques to date ancient civilizations, once navigation by stars was invented. Not sure if this is a use case for some of these packages. I did not personally try to reproduce the math. If you find problems with the math I can bring it to the editor’s attention.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [August 23, 2023, 5:10pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/86 "2023-08-23T17:10:43Z")

</div>

> [@cstjean](#):
>
> why not make DateTime parametric?

I wonder if that would be considered a breaking change. Maybe not technically breaking, but it could slow down code that assumes `DateTime` is a concrete type. For example, this struct would go from having concretely typed fields to having abstractly typed fields:

```julia
struct A
    t::DateTime
end

```

---

<div class="post-metadata">

**Author:** ![Jordan\_Cluts](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jordan_cluts/32/13753_2.png) [@Jordan\_Cluts](https://discourse.julialang.org/u/Jordan_Cluts)\
**Post date:** [August 23, 2023, 5:27pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/87 "2023-08-23T17:27:14Z")

</div>

I’ll toss in my 2 cents here. One of the threads throughout this conversation has been (both explicitly and implicitly at various points) that maybe more precise DateTimes, wider epochs, etc. should be done in a package for users that need that functionality. I don’t disagree and AstroTime.jl and NanoDates.jl are great packages.

Unfortunately, the structure of Base.Dates makes the implementation of a custom `Dates.AbstractDateTime` subtype very complicated. In particular one common use case many users may want is the ability to parse a string. Base.Dates provides a function for this but it is tailored quite narrowly to only the type in the base package. Extending it leads to various difficult tradeoffs. For more details check the discussions under closed issues in NanoDates.jl.

This means that, unlike other Base types such as AbstractArray, AbstractDict, etc. one cannot quickly and easily create a custom concrete type to match ones needs and/or swap to an alternative implementation when one’s code runs into the limitations of Base.Dates.

So all-in-all a conversation about what Base.Dates should or shouldn’t support involves leaving some functionality up to packages but that functionality is difficult to realize. Therefore either Dates should expand to cover more use cases or be improved to make the implementation of alternate DateTime types plausible.

\*edit: fixed a typo

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [August 23, 2023, 9:00pm UTC](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949/88 "2023-08-23T21:00:26Z")

</div>

> [@cstjean](#):
>
> why not make DateTime parametric?

Or make a new struct:

`PreciseDateTime{T}`

Which is parametric but leave the existing DateTime alone… But make adding a `Nanosecond(1)` to a DateTime promote to PreciseDateTime{Int128} by default?

[Previous page](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949.md?page=3)

[Next page](https://discourse.julialang.org/t/datetime-arithmetic-on-microsecond-or-smaller-scale/102949.md?page=5)