This isn’t quite right. CockroachDB and Yuggabyte both use conventional clocks to reliably order events. Spanner uses GPS to do so (conventional time stamp but maybe not technically a conventional clock).
* Spanner creates very tight bounds on the clock synchronization (through atomic clocks/GPS) and ... just waits out the length of the bound.
* CockroachDB seems to do lamport-clocks-with-real-timestamps for linearizability (track the highest seen timestamp for causality chains). For preventing consistency violations with reads, they also track the bounds on the clock and potentially attempt to read again.
So they approve of the overall message (of not just comparing conventional timestamps) but work around those using the uncertainty of the clocks they have available.
My professor used to introduce logical clocks with "as we usually can't use atomic clocks on all nodes" because of that.
https://muratbuffalo.blogspot.com/2022/11/practical-uses-of-...
Seems the bottom line is: "Since clock synchronization can fail occasionally, it is most desirable for algorithms to depend on synchronization for performance but not for correctness."
ts = get_atomic_time_with_no_delay_because_this_node_has_an_atomic_clock();
do_stuff_with_time(ts);
is really broken if you have to wait for garbage collection, wait for your process/thread/coroutine/... to be scheduled, wait for a nearly invisible pause as a cache line is refreshed, .... The time you're using the atomic timestamp is at some future moment, and if that delay matters for your algorithm you need to track your uncertainty just like Spanner/Cockroach do (you just get significant benefits because the uncertainty is low).
Last time I worked on something like this was 20 years ago, but it was a Pentium box. The clock firmware to converted a 5MHz sinewave output to 1Pps, but could also output a synthetic UTC through LAN in cases where there was no other connectivity. I guess I'm saying this is really specialized code for whatever hardware/OS you're using.
(edit) Here's some documentation for a very similar box: https://www.manualslib.com/manual/1281621/Symmetricom-5071a....
In a distributed system, the absolute value of the time is important, since that's what your transaction timestamps are based on. There is a lot to do to ensure that your time offset from UTC ends up being correct; antenna cable length compensation, oscillator quantization compensation, etc. These are not strictly necessary for Spanner but can make microsecond-level differences which are quite noticeable.
"The API directly exposes clock uncertainty, and the guarantees on Spanner’s timestamps depend on the bounds that the implementation provides. If the uncertainty is large, Spanner slows down to wait out that uncertainty. Google’s cluster-management software provides an implementation of the TrueTime API. This implementation keeps uncertainty small (generally less than 10ms) by using multiple modern clock references (GPS and atomic clocks)."
Basically, there is some tradeoff between clock synchronization perfection and transaction processing speed that can be made. You can build dedicated hardware that's clocked by a GPSDO and provide an API to get hardware timestamps, and maybe process transactions a little faster. Your good old C++ program running on Linux with an internal oscillator adjusted by NTP with a GPS PPS input still beats communicating between datacenters to agree on event ordering, though. Light is slow!
Only for a co-variant data cohort is total temporal order necessary for correctness. It doesn't need to be system wide. Distinct (data independent) process groups can be partially ordered.
There really is no such thing as "now" outside of a shared frame (or intersecting frames) of observation.
Anyhow, here's a nice summary: http://muratbuffalo.blogspot.com/2014/07/hybrid-logical-cloc...
> perfectly synchronized clocks are a holy grail of sorts for distributed systems research. They provide, in essence, a means to absolutely order events, regardless of which node an event originated at
two events that occur on two physically discrete nodes at exactly the same time have no well-defined order, even if their clocks are synchronized
> before a node is allowed to report that a transaction has committed, it must wait 7ms. Because all clocks in the system are within 7ms of each other, waiting 7ms means that no subsequent transaction may commit at an earlier timestamp, even if the earlier transaction was committed on a node with a clock which was fast by the maximum 7ms. Pretty clever.
so sending information between two antipodes on earth takes 66ms in one direction, 132ms round-trip, minimum. the speed of light dictates this lower limit
if two transactions are made on those two nodes at exactly the same time, there is no objective order between the two. you can choose an order, but only with knowledge of both transactions, and that information physically cannot traverse space-time in 7ms
so it's really not clear to me how this works. as an optimization, sure -- maybe there is some optimistic concurrency control that will fail transactions that conflict in the way i've described?
if you want to make assertions about order between events from a set of nodes, and you want to use per-node physical clocks to determine that order, then even if those clocks are perfectly synchronized, order can only be decided when the light-cones of all nodes intersect, which is the maximum distance between any two nodes times the speed of light
7ms works for up to 2098km, that's in the best case
That said, the analogy with causality in relativity and light cones etc is useful, and it’s no coincidence that Lamport wrote a book on general relativity before he turned to distributed systems.
Isn't the whole point of special relativity that a Newtonian universe with a finite speed of light isn't Newtonian?
What makes relativity special is that everybody agrees on the speed of light. So Alice is passing Bob at say 0.1% of c, she detonates a big firecracker or other bright sudden event, the light from that event forms a big bubble that expands away from her in all directions at speed c.
If you imagine that light is like sound, that it propagates through a medium called the lumineiferous ether, then maybe Bob is at rest relative to the luminiferous ether. If so, he thinks that the bubble is centered on a fixed point in space, and Alice is just a smidge closer to one of the sides of the bubble than she is to the other. Alice, on this account, sees the center of the bubble drifting “backwards” relative to her motion and agrees that she is closer to one edge than the other. If they were right next to each other when the light burst went off, Alice agrees that Bob is the true center of the light bubble.
Special relativity says that there is no ether: Bob thinks that this expanding light bubble is indeed expanding from a fixed point and that Alice is closer to this side than that, and he's right... but Alice thinks that this bubble is expanding out uniformly with herself at the center, and she is also 100% right. According to her, Bob is closer to one side of the bubble than the other, and Alice is the true center.
This has a bunch of consequences. The first one is that it makes the Zeno paradox into a real life thing. To outrun a light beam, you first have to accelerate to half the speed of the light beam, at which point if you look to see how fast it's going away from you, it is receding at speed c still, so you accelerate again to half the speed of the light beam and it is still moving at speed c away from you, and you can never catch up to it. Bob watching this must agree that Alice never catches up to the leading edge, even though he sees her dumping tremendous amounts of energy into her motion. So nobody can accelerate faster than the speed of light, it takes infinite energy to get there. (The converse of this has now become engineering reality, all of our particle accelerators dump huge amounts of energy into small particles and operate under the simplifying assumption that they all max out at speed c.)
The Newtonian universe with finite speed of light is the first situation, the relativistic universe is the second.
in that Newtonian universe, good enough clocks and time stamps could be used to break the tie. good enough clocks will never be enough in our universe, there is no canonical way to say of two events separated in a space like way is first. in a newtonian universe you can.
If a system never orders events wrong but will immediately stops processing events when enough number of nodes have their clocks out-of-synced, which is the case with all systems using conventional clocks, can it still be considered reliable?