While this seems like a reasonable solution at first, it doesn't fix the vulnerability. It should be obvious when you consider that the attacker already deals with natural variation in network speeds and server load.
Turns out it's very straightforward to use statistical analysis to get rid of the noise and adding a little more doesn't do anything useful. The most reliable way to throw off the attacker is to make sure you do the same amount of computing in every case.
"It's stuff like this that makes building secure systems very hard."