But as performance gets worse, the ping being slow matters less, relatively, right?
The ping is just being used to measure of the foundations of the system and the OP is pointing out don't expect to see a 'high performing' system when the ping has identified the foundations are broken.
If you fix the ping and then system is automatically fixed.