Apologies, I was under the impression it was a significant cost. Do you have any good analysis of where Rust compiler time goes you could link to?
Here is an older article that compares compilation times between multiple different libraries, where one of them does spend a significant (and majority of the) time borrow checking: https://wiki.alopex.li/WhereRustcSpendsItsTime