Convert Dates.DateTime to int64 since epoch

What’s the correct way of converting DateTime to number of milliseconds since epoch? The function datetime2unix returns float.

julia> using Dates

julia> n=now()
2020-07-09T22:41:28.191

julia> u=Int(floor(datetime2unix(n)))
1594334488

julia> unix2datetime(u)
2020-07-09T22:41:28
u=floor(Int, datetime2unix(n))

First off, that doesn’t seem efficient, as internally DateTime is an Int64, so you’re taking an Int64, converting it to Float, rounding it, and converting it to a different Int64. Second, it’s not even clear to me that this is correct for all values, given the rounding. Certainly the rounding loses you the milliseconds.

julia> now().instant.periods.value
63729997158785

That’s an int, but it’s not since epoch.

yeah because you need to subtract epoch

1 Like

Damn that’s a good find, thank you.

So the only reason why datetime2unix returns float is because of that /1000.0. Crazy. Implementing my own datetime2epoch now which returns int milliseconds :smiley: Thank you

1 Like

Freeman. Please do not keep all of us in suspense.

Please show us your source code for datetime2epoch function

Hahah well the point was just that if you don’t demand that the result to be in seconds and you’re ok with getting the result in milliseconds (or even better, nanoseconds[1]), then you don’t need the /1000 bit, or the floats or the roundings.

So now I’m using:

datetime2epoch(x::DateTime) = (Dates.value(x) - Dates.UNIXEPOCH)*1_000_000

This returns an int with the number of nanoseconds since epoch.

[1] I say that nanoseconds is even better because it’s the internal representation of Python’s pd.Timestamp. It’s some little interop you gain with no extra effort. This can be seen to be correct in the following way:

julia> x = DateTime("2020-06-01T00:01:02.693", DateFormat("yyyy-mm-ddTHH:MM:SS.sssZ"))
julia> datetime2epoch(x)
1590969662693000000
>>> import pandas as pd
>>> pd.Timestamp(1590969662693000000)
Timestamp('2020-06-01 00:01:02.693000')

A more general version could look like this:

datetime2epoch(::Type{Millisecond}, x::DateTime) = Millisecond(Dates.value(x) - Dates.UNIXEPOCH)
datetime2epoch(::Type{Second}, x::DateTime) = Dates.value(datetime2epoch(Millisecond, x)) / 1000
datetime2epoch(::Type{T}, x::DateTime) where {T <: TimePeriod} = datetime2epoch(Millisecond, x) |> T

Usage:

>>> dt = DateTime("2020-06-01T00:01:02.693", DateFormat("yyyy-mm-ddTHH:MM:SS.sssZ"))
>>> datetime2epoch(Second, dt)  # float - seconds can be noninteger
1.590969662693e9
>>> datetime2epoch(Nanosecond, dt)  # Millisecond and smaller give the corresponding period type
1590969662693000000 nanoseconds
1 Like

Hmmm so if you use ::Type{Second} you get back type Float64 but if you use ::Type{Millisecond} you get back type Millisecond? :thinking: Doesn’t that seem surprising from a user perspective?

Unfortunately, yes - this is because Second and other types do not support floating point values.

If Second doesn’t support floating point values in that case wouldn’t it make more sense to floor it or round it? I mean, DateTime has more time resolution than Second, and the user must be aware of this. When the user writes ::Type{Second} what does he expect? Does he expect the correct type with second precision loss, or does he expect a different type with floating precision loss?