The main "difficulty" is that "Rdtsc" has itself a large execution time, and that modern processors have a very strange definition of "execute".
With retirement queue sizes ranging from 300 to 600 (depending on architecture), your singular CPU-core could very well be "executing" 600+ instructions simultaneously, including "rdtsc".
Under such conditions, the "definition" of "get the current time" is quite ambiguous.
-----------
This isn't a problem with bandwidth. But it makes measuring "latency" very difficult, especially because out-of-order execution means that "proper" latency is largely ignored.
-------
Because of the out-of-order execution models of modern processors, "reciprocal bandwidth", or "instructions-per-clock" tends to be more important than latency proper.