I believe the core of the problem is that it has to reparse the code to pattern match for the macro.
One egregious patter is tt-munchers [1] where your macro is implemented recursively, requiring it to reparse the source on each call [2].
In one of my projects, someone decided to wrap a lot of core functions in simple macros (ie nt tt-munchers) to simplify the signatures. Unlike most macros which are used occasionally and have small inputs, this was a lot of input. When I refactored the code, I suspect dropping the macros is the reason CI times were cut in half and a clean `cargo check` went from 3s to 0.5s.
[0]: https://nnethercote.github.io/2022/04/12/how-to-speed-up-the...
[1]: https://veykril.github.io/tlborm/decl-macros/patterns/tt-mun...
[2]: https://github.com/dtolnay/quote/blob/31c3be473d0457e29c4f47...