Easel constantly synchronises the clocks (there's an interesting algorithm for this which I will write up at some point). It also adaptively assigns two different kinds of delay to every client - command delay and display delay. Command delay is related to how much lag you are introducing into the game. Basically people take on their own lag. It can be different amounts for different people in the game. The display delay is where the rollback netcode kicks in. It keeps track of how much rollback your computer can handle imperceptibly. If your computer can't handle it, then you won't get as much rollback (and will just experience more input latency). But in either case, whatever number it picks, it should be smooth.
Here's an example of the constant one sided rollback: https://www.youtube.com/watch?v=aSB_JlJK_Ks
and an example of how players were aware that mashing caused frequent rollbacks: https://youtu.be/_jpg-ZiE70c?t=105
Eventually PC players learned that they could "fix" this by alt-tabbing out of the game, taking away dedicated GPU processing so they could be the peer that was falling behind.
As far as overestimating the base model PS4, I've shipped rollback multiplayer games that support 32 players that run on Android Web Browsers. I would love to see a frame profiler of SFV. I have years of experience optimizing multiplayer games to run on far worse platforms than the PS4. 16ms is a lot of time.
I bet I could make SFV's simulation run in less than a millisecond on an Android phone from Walmart. It couldn't render on that device, but it could run the raw simulation.