With my numbers, you'd have 50 ms total. The bluetooth delay of 50 ms has two significant figures in it at best, and so when we add the 0.002 ms (2 microseconds), that's just noise.
Are you really equating people's perception of tens of milliseconds of latency with their perception of nanoseconds? Is it surprising that people would notice a 50 ms slowdown more than a 50 nanosecond slowdown?