servers should always be UTC
There are terminal servers, for example, which are basically workstations. There are servers configured by other people for local time, and we may need to configure our own servers into its network.
There is server software out there that uses the host timezone. Heck, huge vendors do this regularly. This happens in log files, in schedulers, in notification emails, and so on. People want to see their log timestamps in their local time, they want to schedule in local time, and they want their emails to show events in their local time.
Yes, optimally, all times should always stored in UTC or UTC+offset formats, then displayed in local time only "at the end client". Generally, this is actually what happens. Windows, MacOS, UNIX, Linux, and Android all store the system clock as UTC and all system APIs use this.
So in some sense, we are all already using UTC on our servers. The time zone setting doesn't change the system clock to something other than UTC, it just tells user-mode software how the human users expect to see time formatted for display.
The problem is that server software especially is terrible at this. Just... bad. The worst offenders are text-based log files, which almost always get a formatted timestamp with an unspecified time zone. Could be UTC, could be the server's local time, could be Mars time, who knows?
In summary: don't blame system administrators for correctly configuring server settings. Blame the lazy software developers who can't be bothered to use ISO timestamps that specify time zones unambiguously, irrespective of the time zone setting.
Next thing you'll tell me to stop using space characters in file names...
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation] "RealTimeIsUniversal"=qword:00000001
We share this planet with people that don't use UTC. That's just the way it is. If some computers are set to time zones other than UTC, then non-UTC time zones must be correctly handled, even "servers".
This is just patronizing hand-wringing; well done you.
For global systems, a single time convention is the basis of co-ordination. So many people on this thread have seem to have a "pets not cattle" viewpoint, it is quite concerning.
For a terminal server it is not uncommon when different users connect from places with different time zones. It would make sense to have TZ in a user profile but keep UTC as a server system locale.
As far as practical use of UTC vs local time: I am a developer that happens to work with customers directly. It is difficult enough to get customer to provide timestamps for issues they report in local time, I cannot imagine they would be able to convert to UTC.
Love the local time!
The only question is, is the implementation user focused or developer prioritized
What exactly do you mean by this? As in, what part of your software stack is doing this. IIRC if you set your OS to be in UTC then anything at the OS level will only speak UTC. Databases and your own software will do what they are told to do but systemd etc. will all speak UTC only. Even running date in the shell should give you a UTC timestamp.
I'm not even sure what you mean by the OS speaking a certain timezone. Basic "time()" will return seconds from the epoch UTC regardless of your system timezone. Things that care about timezones will use relevant conversions.
Apart from this issue, I’ve never once see an error due to the tz presentation settings on a server, and I have spent 20 years running many thousands of servers at a time. shrug.
Let them have to adjust in grafana/kibana if it helps them sleep I guess..
> but I can’t imagine it arbitrarily converts anything away from UTC time if /etc/localtime is set to UTC.
It really depends on which part of systemd you mean. There's lots of code in there and quite a few bits deal with timezones explicitly. (Since it sets it for users to begin with)
But for basic display it sure uses TZ and localtime as set by the user - and it can be different than the system default. https://github.com/systemd/systemd/blob/cccc14c5a88f000931c6...
A few decades pass and we now all have highly accurate "clocks" that we carry around in our pockets that receive GPS signals and sync their idea of time to what the satellite thinks. Sounds great, right?
Except, humans don't actually use atomic clocks as the basis for their timekeeping. We look outside and expect the time on the clock to mostly match with the position of the sun in the sky.
Once leap seconds became entangled in them, the whole thing basically a toxic pile of radioactive waste. Timezones are now being updated multiple times a year since the internet means we can update them at will. The whole thing is now a giant mess of toxic radioactive waste that can never be correct for any extended period of time.
We, the software industry, did this to ourselves. We built this, we have to live with it, and it's never going away.
It really isn’t that hard. What makes you pull your hair out is dealing with 3rd party code that handles it incorrectly. Do everything internal in UTC or always attach a timezone to dates, your choice.
When you cross the boundary either ingesting naive dates or displaying them read the system time zone again and convert.
I mean it’s kinda annoying but “naive” times is the time equivalent of storing arrays without their length. I can’t fathom why we even allow it.
Webapps should be timezone aware via the browser or user preferences, internal timezone should be whatever makes sense for logging et al.
[0]: https://gist.github.com/stephanGarland/b7cdd963e0ac53ea42f8e...
[1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1019716
I strongly prefer everything in UTC and changing time zone is an issue for only the vary top layer of the display/user interface/frontend whatever if absolutely needed - generally libraries designed for displaying have much more robust timezone handling than just random daemons and backend software.
All of the servers I run are located in Toronto, Canada: why do I care about UTC? What would having all of them being UTC get me?
A few jobs ago I took care of servers that were in: California, Ontario, Quebec, Switzerland, Singapore. UTC made sense there.
If I look at something now (at Mar 25 21:09) the logs would say "Mar 26 01:09" which is not "now" in my mind.
† Though I've been toying with the idea of getting a GMT watch "for fun". However I'll probably go with a chronograph—more useful feature on a day-to-day basis. Not as many choices if I want to get both complications though.
Maybe your logs mention the timezone / UTC offset? Otherwise it would seem possible to get confused.
But I do sometimes have to look at the logs on some random Tuesday, mid-morning, when someone says they can't log into our SSH bastion hosts (but it worked yesterday), I can see there's failed password attempts at 09:59, 10:03, and 10:04 (versus 13:59, 14:03, and 14:04 (i.e., UTC+4), which I would then have to do mental math to bring back to local time), and so it turns out they entered their password wrong a few of times so fail2ban blocked their IP
Wallclock time is a human construct created for human convenience. Having UTC when all my servers are in America/Toronto is not convenient for me.
Having UTC when all my servers are in America/Toronto is not convenient for me.
I will perhaps consider doing it at that point, but in ~20 years of being a sysadmin I have never needed to, so… ¯\_(ツ)_/¯
Therefore I optimize for the common case: looking at logs when events do not straddle DST changes.
We could use both dates in logs, but that would eat an unreasonable amount of columns and confuse naive greps. We could also use non-standard time formats, e.g. <localtime8601>[<dst switch warning>] <unixtime>, but standard log handling tools aren’t built for it.
* https://en.wikipedia.org/wiki/Energy_Policy_Act_of_2005#Chan...
From a Unix perspective (Solaris, Linux, IRIX), it was fine. And Ontario has provisionally passed getting rid DST changes (sadly in favour of going to year-round DST), and IMHO it will be fine again if it is ever finalized:
* https://toronto.ctvnews.ca/ontario-passes-legislation-to-mak...
So long as you keep your TZ database up-to-date local time is likely to be not-so-problematic. Once you have that one server in a closet that didn't get updated, all bets are off.
It's only one less thing to worry about if you worry about it in the first place. I do not.
> I'd flip the argument on its head and posit that there's no compelling reason to use anything other than UTC.
There is a compelling, or at least useful, reason for me; from another reply I did:
Using UTC when all of my servers are in one timezones actually decreases clarity, as I live in America/Toronto with my daily existence, and which is what my laptop and (mechanical) wristwatch are set to. If I less(1) the logs I now have to do mental math as to "when" things happen because the hour number it says in the logs does not match my human reality.
If I look at something now (at Mar 25 21:09) the logs would say "Mar 26 01:09" which is not "now" in my mind.
[0]: https://www.ola.org/en/legislative-business/bills/parliament...
Local time is governed by local authorities; UTC, much less so.
I mean, clocks go out of sync, every few (days? Weeks?) I assume the ntp client has adjusted actual local time on your server; not just the time zone your server translates it to when you run “date” we are talking about here…
If it can handle that, it can handle a tz with dls time.
Usually being a few seconds ahead is compensated by running the clock slower for some time. Showing the clock to compensate a whole hour breaks many kinds of assumptions. But jumping the clock actually back an hour breaks even more assumptions that look entirely sane.
Better off without it.
Eliminates all the software bugs due to non-monotonically increasing time.
If you get bought by a company in the EU/UK or vice versa then it ultimately makes it easier to deal with time as you become part of a global company.
For a system to not boot in a particular time zone is a serious bug, but you should not blame the problem on the time zone, you should blame the problem on whatever is broken in some critical piece of code that probably should not be using local time.
I mean, yes, you can do that, but don't pretend that a program not handling unicode is the user's fault for typing unicode (s/unicode/timezones, s/typing/using).
EDIT: I'm just saying that you can't blame the user for using the features that come with the system.
For anyone curious:
> On the flip side, there’s one border where traveling north or south puts you ahead or behind just 15 minutes. That would be the border between India (UTC +5.5) and Nepal (UTC +5.75).
* https://qz.com/357697/time-zone-deviants-part-i-the-stranges...
It’ll smoke out all sorts of fun.
> you could say servers should be airgapped because security is hard
Many effectively do, and I wish more would do, as a default starting point. Maybe not physically air-gapped but firewall all incoming and outgoing then selectively allow packets through as specifically needed.
Translating to timezones: default to UTC and configure hosted software to display local timezones as needed. Only have the system in a different TZ if something won't work properly otherwise (including it can't be configured to display in something other than the system TZ) or is date sensitive and buggy in a way that it gets confused about yesterday/today/tomorrow when local midnight and UTC midnight differ.
Also if considering resources used by people in multiple timezones, you have to pick something and translate for others, you might as well pick UTC as some things assume it.
> Or that you should only type ASCII because Unicode is hard.
Again yes, by default, and always for things like hostnames, and adjust at needed or when you know Unicode is supported end-to-end (or that where it isn't, something just looks wrong and firefighting that is preferable to not sticking to ASCII in the first place). Though it helps from my point of view that the most obvious omission from ASCII in my locale is our currency symbol.
I say the same for spaces in filenames too: while you perhaps shouldn't have to avoid them, in practise you are setting yourself up for potential trouble later if you don't.
> EDIT: I'm just saying that you can't blame the user for using the features that come with the system.
I agree completely.
But, for instance, while I won't victim blame when someone accidentally leaves doors unlocked and has things stolen, I do make an effort to be damn sure my doors and windows are secure and recommend others do too. There is a difference between victim blaming and recommending preventative defaults because you know there is shit code it there.