Agreed. They discuss this in the chubby paper as well.
There is a grace period where the master is in jeopardy of failing over vs failing over straight away. I left this out for simplicity of the state machine but I pointed to the chubby paper as a true reference.
They also mention the fact that client and master clocks must advance no faster than a known constant factor.