Does anyone have a good explainer for how the NTP protocol works? I can't quite wrap my head around how you could possibly synchronize two machines in time over a network with unknown and unpredictable latency.
Specifically on the latency question, have a look at https://stackoverflow.com/a/18779822 for a basic explanation. tldr, once you allow for two-way communication you can start to factor out the network delay.