You are assuming not only that an attacker can see all packets all the time, but also at all points in the network, which is unlikely in practice.
Why is it unlikely for them to have that kind of temporal resolution? I would assume the opposite. It seems like it would be trivial to save timing information along with packets. And the timing data must be relatively small compared to whatever data they're saving on the packets, so the added storage requirement is not really a concern.
This isn't to say they couldn't specifically target an endpoint for monitoring, however. Perhaps I was unclear about that.
What a lovely euphemism.