Unix2datetime giving wrong time and date?

phantom · May 11, 2023, 5:02am

Hi I’m sure I’m missing something simple here but when I test for the time of a file that was created on 5-10-2023, I get a result

julia> unix2datetime(mtime("/Users/Desktop/Test Data 2023 5 10.numbers"))
2023-05-11T04:27:58.075

julia> unix2datetime(ctime("/Users/Desktop/Test Data 2023 5 10.numbers"))
2023-05-11T04:27:58.080

I suspect this might be a TimeZone issue however both now() and localzone() yield the correct time and result (which are consistent with the settings on my computer system.) So I’m not sure why this is returning something different. Any thoughts on what the issue is would be greatly appreciated, Thanks!

laborg · May 11, 2023, 5:42am

You write that you have “created” the file on 5-10-2023. Could you check ctime instead of mtime? You might have accidentally modified the file today?

phantom · May 11, 2023, 6:09am

Hi thanks for pointing out that distinction! I edited the post to include both ctime and mtime. I just created the file on the 10th for testing purposes so I hadn’t modified or even opened it afterwards.

piechologist · May 11, 2023, 8:46am

mtime and ctime return the epoch seconds which are always in UTC. Converting with unix2datetime yields the DateTime in UTC.

You can always use UTC (e.g. call now(UTC) if you want to compare an mtime with the current time) or you can use the TimeZones.jl package to convert to your local time.

You can print the local times with stat(foo), e.g.:

StatStruct for "foo"
   size: 10469 bytes
 device: 16777220
  inode: 438829
   mode: 0o100644 (-rw-r--r--)
  nlink: 1
    uid: 501 (me)
    gid: 20 (staff)
   rdev: 0
  blksz: 4096
 blocks: 24
  mtime: 2023-05-07T15:48:46+0200 (3 days ago)
  ctime: 2023-05-07T15:48:46+0200 (3 days ago)

phantom · May 11, 2023, 10:12am

Thanks for clearing that up! I thought that mtime would be based on the system time. I’m still having some trouble converting to local time on my system.

Is there a way to get TimeZones to display the actual time and date as opposed to UTC +/- adjustment?

With the TimeZone example provided in the docs I get the following.

ZonedDateTime(unix2datetime(mtime("/Users/Desktop/Test Data 2023 5 10.numbers")), tz"Europe/Warsaw")
2023-05-11T04:27:58.075+02:00

However I am unable to convert it to a normal DateTime format. If I try the conversion is dropped.

Dates.format(ZonedDateTime(unix2datetime(mtime("/Users/Desktop/Test Data 2023 5 10.numbers")), tz"Europe/Warsaw"),"yyyy-mm-dd HH:MM:SS")
"2023-05-11 04:27:58"

If I try to convert with other TimeZones provided via timezone_names() I get the following error.

julia> ZonedDateTime(unix2datetime(mtime("/Users/Desktop/Test Data 2023 5 10.numbers")), tz"US/Mountain")
ERROR: ArgumentError: The time zone "US/Mountain" is of class `TimeZones.Class(:LEGACY)` which is currently not allowed by the mask: `TimeZones.Class(:FIXED) | TimeZones.Class(:STANDARD)`

ZonedDateTime(unix2datetime(mtime("/Users/Desktop/Test Data 2023 5 10.numbers")), tz"US/Pacific")
ERROR: ArgumentError: The time zone "US/Pacific" is of class `TimeZones.Class(:LEGACY)` which is currently not allowed by the mask: `TimeZones.Class(:FIXED) | TimeZones.Class(:STANDARD)`
Stacktrace:

However both “US/Mountain” and “US/Pacific” are listed in timezone_names() under the latest version of TimeZones 1.9.2 . Just wondering what else I was getting wrong here.

piechologist · May 11, 2023, 1:20pm

ZonedDateTime() assumes you’re passing a local time. I guess you want:

# Get a ZonedDateTime with time zone information:
ZonedDateTime(unix2datetime(mtime("foo")), tz"Europe/Warsaw"; from_utc=true)

# Turn it into a simple DateTime with no time zone information:
DateTime(ZonedDateTime(unix2datetime(mtime("foo")), tz"Europe/Warsaw"; from_utc=true))

If I need more than just local time in a project I stick with ZonedDateTime or convert everything to UTC. Mixing some DateTime objects that are in UTC with others in local time will bite you sooner or later.

Regarding the ArgumentError, time zones like “US/Pacific” may be ambiguous and thus are marked as legacy. Sometimes you see PT and it may be PST or PDT. It’s better to use specific locations like “America/Los_Angeles”.

phantom · May 11, 2023, 2:21pm

Awesome thanks so much! I was very confused about how to use this package. Just curious how did you find out about the additional keyword arguments in ZonedDateTime like from_utc? I’m probably missing something but I don’t recall seeing it in the documentation. Also why does

ZonedDateTime(now(), tz"UTC")

give the same time as now() - i.e. it returns the local time on my computer rather than the current UTC time whereas

now(tz"UTC")

gives the current UTC time? I would have thought they would return the same thing?

piechologist · May 11, 2023, 3:36pm

You’re right, it’s missing in the introduction but mentioned further down in the Public API. Usually, I read the docstrings inside the REPL (julia> ?ZonedDateTime).

ZonedDateTime(now(), tz"UTC") takes the DateTime as it is and just attaches the time zone UTC. Think of it as slapping a label on it.

# Slaps the time zone on my local time:
julia> ZonedDateTime(now(), tz"America/Los_Angeles"; from_utc=false)
2023-05-11T17:20:26.972-07:00

# Converts my local time to UTC and then slaps the time zone on it:
julia> ZonedDateTime(now(), tz"America/Los_Angeles"; from_utc=true)
2023-05-11T10:20:30.155-07:00

Watch the difference between DateTime and ZonedDateTime:

# Local time:
julia> now()
2023-05-11T17:23:33.166

julia> typeof(now())
DateTime

# UTC:
julia> now(UTC)
2023-05-11T15:23:49.270

julia> typeof(now(UTC))
DateTime

# UTC, this time we implicitly call now() from the TimeZones.jl package:
julia> now(tz"UTC")
2023-05-11T15:24:22.048+00:00

julia> typeof(now(tz"UTC"))
ZonedDateTime

I agree that the whole stuff is confusing at the beginning but it’s worth reading into it. Time zones are very tricky and I’m thankful for TimeZones.jl making them easier to handle.

stevengj · May 11, 2023, 3:50pm

Note that this calls Libc.strftime("%FT%T%z", t), which in turn calls Libc.TmStruct (which calls localtime_r) to convert the Unix time to a struct tm. So, another way to get the DateTime in the local timezone, without using the TimeZones.jl package, would be to do:

DateTime(Libc.TmStruct(mtime(filename)))

It would be good to add this as an option to unix2datetime: [Request] Dates.unix2datetime with local time option · Issue #49297 · JuliaLang/julia · GitHub

piechologist · May 11, 2023, 4:55pm

That’s handy! Not nick picking, but there’s one caveat:

julia> t1 = DateTime(ZonedDateTime(unix2datetime(mtime("foo")), localzone(); from_utc=true))
2023-05-11T18:34:30.174

julia> t2 = DateTime(Libc.TmStruct(mtime("foo")))
2023-05-11T18:34:30

julia> t1 == t2
false

The C call omits the milliseconds which may cause trouble when comparing time stamps or searching for older or newer files – speaking from experience . Once I had to debug a similar issue in a shell script and it’s not obvious if some tools don’t report the milliseconds at all.

stevengj · May 11, 2023, 5:34pm

Would be easy to add them back in:

function unix2datetime_local(t::Real)
    s = floor(t)
    ms = (t - s) * 1000
    return DateTime(Libc.TmStruct(s)) + Millisecond(round(Int, ms))
end

phantom · May 12, 2023, 4:52pm

Thank you both for providing such thorough and perspicuous answers! I’m not sure how to select a single solution that would best serve the interest of the forum because each post was so helpful. Anyone as confused as I was would have to understand the distinction between mtime and local time prior to converting said times with theTimeZones.jl package or the unix2datetime_local() solution so I defaulted to that.