Your server has NTP. Your client likely, too, if it has enough network to connect to the server. If it does not use NTP, it's easy to set the time within the minute or so required manually.
Would I employ this for a critical commercial server? No. Would I do so for my private server? Absolutely, without hesitation. If it really loses track of time so badly (hasn't happened yet to my knowledge), I can just log in through virtual console.
[1] I'm excluding drift and inaccuracy, since from my anecdotal experience they are usually not that bad.