you can get it down to nearly zero using OpenWRT and configuring SQM. Use https://www.waveform.com/tools/bufferbloat to measure and tune until you get an A+ result.
Bufferbloat is good thing to fix in general though and highlights why one should worry less about "how do single ICMP packets behave" and more about "how does actual loaded traffic of the protocol type I'm using behave in real usage". The results are often staggeringly different.