Hacker News Reader

Comment by Misdicorl | Hacker News Reader

Almost all the edge cases with time have to do with localization.

Build your time library on (un)signed 64 bit integers representing the number of nanoseconds since the utc epoch. Adjust above sentence to reflect the level of precision and range your use case needs. You are now done for 80% of use cases (perf timing, logging, timeouts, event storage, event ordering within jitter).

If you need to parse/display for humans or have something happen at a particular time in a particular timezone, things get gross. But that's no different than any other situation where you eventually have to interface machine data with humans. Either it's your particular expertise or it's a distraction and you should use someone else's solution.

View on HN

> Build your time library on (un)signed 64 bit integers representing the number of nanoseconds since the utc epoch.

You poor sweet summer child. Has no one told you about the leap seconds yet? Unix time is not the number of seconds since epoch. It deliberately excludes leap seconds, which happen unpredictably whenever scientists measure the Earth as having spun at a different enough speed for long enough.

Time is fucked on every level:

- Philosophical: What is time? We just don't know.

- Physical: Turns out there is no such thing as simultaneity, and time flows differently at different locations. Time may be discrete at the Planck level, but we don't really know yet.

- Cosmological: The Earth does not rotate at a constant speed, the Earth does not orbit the Sun at a fractional component of its rotation, and the Moon does not orbit at even ratio either.

- Historical: Humans have not used time or calendars consistently.

- Notational: Some time notations are ambiguous (e.g. during daylight savings transitions) and others are skipped.

- Regional: Different regions use subtly different clocks and calendars.

- Political: Different political actors choose to change time whenever they feel like it with little or no warning.

- Religious: Many religions come with their own system for timekeeping, and people don't like when outsiders impose other systems.

"You poor sweet summer child"

haha. Google is way ahead of you and your "leap seconds".

"Since 2008, instead of applying leap seconds to our servers using clock steps, we have "smeared" the extra second across the hours before and after each leap. The leap smear applies to all Google services, including all our APIs."

https://developers.google.com/time/smear

...now write code to convert Google time to any other random type of time.

Both

https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd...

and https://en.wikipedia.org/wiki/Unix_time

indicate that leap seconds are not included in Unix time as seconds-since-the-epoch. Leap seconds are included in UTC, and thus the Unix time appears to skip a second relative to UTC.

> Unix time is not the number of seconds since epoch

I assumed that is (part of) why he ended up wanting to build his own. The way UNIX time handles leap seconds was arguably a mistake, GPS (and Galileo) time does it right; have the leap second information in a separate field. So as a time scale I imagine OPs one would be similar to GPS but with different epoch. Of course everyone loves having informally defined ad-hoc timescales around

> Unix time is not the number of seconds since epoch

In fairness, he said to count the number of seconds since the Epoch. That is independent of UTC.

That means using e.g. TAI. Unix time also ignores leap seconds since it counts the number of seconds actually elapsed.

> Unix time also ignores leap seconds

hence it counts the number of seconds that would have elapsed, if they didn't exist. In effect, it's timescale has time that never happened, and time that happened twice.

You can still have problems.

It may be determined that the computer's clock got too far ahead. For example, it booted and you cared about time, but NTP hadn't yet made corrections. Suddenly the time runs backwards.

Leap seconds may get interesting too, especially if you have to predict ahead or if the OS isn't updated often enough. (there is a 6-month warning) If you want to call leap seconds an issue for humans, then you aren't using UTC at all. You're using TAI. Software interfaces often ignore the distinction between UTC and TAI, and even between UTC and UT1, preferring to pretend these issues don't exist. POSIX is in conflict with international timekeeping, effectively requiring that there are zero leap seconds.

> Suddenly the time runs backwards

It is my understanding that the way NTP deamons work is that time is never adjusted backward. Instead, the ticks are "slowed down" on the local machine until it is in sync with the NTP time. However, if the difference is too great then I think NTP deamons might refuse to correct the time all together. So then, if my understanding is correct, your machine is "stuck in the future". But it will never make a jump backwards because of NTP.

However, I am not familiar with the intricate details of NTP so do take this with a grain of salt.

My understanding (i am also not an expert!) is that common NTP implementations will do the slowing down ("slewing") to correct small errors, but will just change the time ("stepping") to correct large errors. It will even do this if that means time going backwards.

Subject to configuration, of course. man ntpd [1] says:

> Sometimes, in particular when ntpd is first started, the error might exceed 128 ms. This may on occasion cause the clock to be set backwards if the local clock time is more than 128 s in the future relative to the server. In some applications, this behavior may be unacceptable. If the -x option is included on the command line, the clock will never be stepped and only slew corrections will be used.

[1] https://linux.die.net/man/8/ntpd

timesyncd also does this for "large offsets", but what "large" is is neither configurable nor documented, but in the source:

    /*
     * Maximum delta in seconds which the system clock is gradually adjusted
     * (slewed) to approach the network time. Deltas larger that this are set by
     * letting the system time jump. The kernel's limit for adjtime is 0.5s.
     */
    #define NTP_MAX_ADJUST                  0.4

> neither configurable nor documented, but in the source:

Build it yourself and pass -DNTP_MAX_ADJUST=XX

Leap seconds are only a problem for wall clocks (localization) and deciding whether to call something utc or tai.

The ntp case is contrived. Either you care and wait until ntp has connected to do your stuff. Or you care and don't let ntp rewind and instead smear. Or you don't care and deal with the consequences.

Leap seconds affect Unix time too, because leap seconds are excluded from "number of seconds since Unix epoch". You can't measure the length of time intervals spanning leap seconds with simple subtraction.

> Leap seconds affect Unix time too, because leap seconds are excluded from "number of seconds since Unix epoch".

They are excluded from "unix timestamps". They would still be part of the number of seconds since Unix epoch.

What does that mean and how would that work?

AFAIK, unix time skips a beat or repeats itself to remain in alignment with UTC, and just keeps chugging along.

"Number of seconds since the unix epoch" describes a physical measurement (that can be related directly to TAI) that would not be affected by leap seconds. You could store that number to allow you to recover TAI timestamps (I'm not aware of any OS that does this, but there's no reason it wouldn't be possible).

Unix time does indeed repeat itself so as to remain in alignment with UTC. But unix time is not the number of seconds since the unix epoch.

Thank you, I think I understand, but disagree on terminology.

Leap seconds are not excluded from number of seconds since Unix epoch, interpreted on a physical, TAI, UTC, typical local time scales.

However, they are indeed excluded from "number of seconds since Unix epoch", when interpreted in unix time. In unix time, those seconds simply don't exists (never happened) and the events of those seconds are smooshed sometime. Unix time representation forms a (quirky) timescale.

I'll quote a few excerpts from 'date' man page of linux, openbsd and for gettimeofday:

Convert seconds since the epoch (1970-01-01 UTC) to a date

Print out (in specified format) the date and time represented by seconds from the Epoch.

The time is expressed in seconds and microseconds since midnight (0 hour), January 1, 1970

Hmm. The first definition I saw of "unix timestamp" explicitly defined it as "86400 * number of days since 1970-01-01 + number of seconds since midnight", which I thought was clever and clear, but I can't find it now. Having looked up the POSIX standard at https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... it seems you're right and I'm wrong; they define "Seconds Since the Epoch" as "A value that approximates the number of seconds that have elapsed since the Epoch...". In my view this is an extremely poor choice of terminology, because "seconds since the epoch" should have its plain English meaning of the number of SI seconds that have passed since that event; defining some "second" that is almost, but not quite, an SI second seems like a recipe for disaster.

Are they truly? There must be a good reason but I can't fathom what it could be

That good reason is that there is no general "formula" for leap seconds, unlike for leap years, they have to be looked up. So you can't do "offline" date calculations if they included leap seconds.

I think that UNIX time stamps are generally a very good approximation, and if you are comparing long enough time intervals for the error to get over one second, and/or that error to matter, you are doing something wrong anyway.

For exact time interval measurements that you have to get exactly right, don't use UNIX time stamps.

The lookup database is incredibly small and updated very infrequently. Certainly smaller and less frequent than tzinfo which is also included in the kernel

AFAICT leap seconds are announced about 6 months in advance (January/June). How would you process times that are to occur after a leap second that may or may not be?

You could store something as '2022-02-02 02:22:22 UTC', but try storing it as seconds since epoch, if you don't know if there will be leap seconds. Not to mention software that may be years or decades old.

Because otherwise you would get annoying off-by-27 errors whenever you did time() % 60.

https://en.wikipedia.org/wiki/Leap_second#Binary_representat...

It depends on the purpose of the app, but you often have to store user entered times with time zone information. The rules for things like daylight savings time can and do change, and you don't want future scheduled events drifting by an hour.

Let's not forget events spanning time zones. Just some random thing that came to my mind: how would you handle calendar entries where half the participants made a DST transition since the entry was created and the other half didn't? This happens for example when half of the team is in the US and the other half in the EU. The transition dates are a week apart.

I remember seeing a thick dead tree type of book with the history of time zones in the US, for figuring out times in historical documents when things were less standardized. It was practically the size of a phone book; I think it probably covered county level history or something like that.

I have a book covering, among other things, Indiana time zones for a few years during IIRC the 1960s.

It’s frankly amazing how much they changed every year. Different counties, and sometimes towns/cities within counties, would jump back and forth year to year. It would have been awful to manage if computers had been more important at that time.

None of these edge cases are solved by using off the shelf libraries instead of running your own.

None of these edge cases have a chance of being handled correctly without an accurate time zone database to detect the problem at all. Good luck maintaining that on the side! I am fairly certain that you would get it wrong. There's a reason why most software relies on zoneinfo.

Yes, as I said above, if you need to localize then you'll need more detail.

When do you not?

When you need some system activity to happen regularly (e.g. every 10 minutes) but not at a specifically human-meaningful time. When you need to know what order events occurred in or how far apart two events are, but don't need to correlate those times with external events (or can easily establish "system" times for any relevant external events). You can cover a lot of cases without having to touch "human time" at all.

I've come to the same conclusion as you: Keep time as a purely monotonic integer for everything and convert that as needed to display to humans.

This also pushes all the madness to the edges and out of the business logic.

That being said, this works well for applications that are not "date intensive" so to speak. If your business logic has to deal specifically with calendar dates, e.g., monthly events, then you have to deal with calendar months and all that this involves, including explicitly dealing with the 29th February.

You want to store both local time (and timezone and/or some proxy for location if possible) and Unix / UTC / integer time. The latter is what your application relies on 99% of the time, but if, say, a given country gets rid of daylights savings time (as Brazil did last year), having the local time is helpful for recomputing your Unix time.

The monotonic integer approach doesn't work for most healthcare applications. Due to safety and compliance requirements we typically need to record both the local time and the zone offset which applied at that instant.

This is surprising for event recording since they should be equivalent but time + zone is strictly more likely to be messed up. E.g 2:30 a.m. is two times during daylight changeover so you must correctly specify est or edt.

You're assuming a source of monotonic nanotime is easy to get and will always be available. This is not the case. As far as I know you have either "wall time" as usually defined, with all the problems associated with clock drift, NTP syncs, computer going into sleep mode, etc, or "nanotime" which is some time delta from some arbitrary reference and which can only be used to compute time differences and not arbitrary instants in time. One can not be converted into the other easily, or at all.

You use the arbitrary reference and sample the wall clock a couple times to interpolate a best effort offset from utc 0.

If you care more than this, you'll be displeased with off the shelf solutions too

Your percentages are backwards, IME. Timeouts and logging and such are definitely not “80% of use cases”. Interfacing with humans and human systems are.

I disagree. Time is pervasive and will exist in every application in some way. It will only matter to the user in a small subset of them

I"m just going to leave this here: https://infiniteundo.com/post/25326999628/falsehoods-program...

Reply on news.ycombinator.com