Apart from rare pathological cases borrow checking is not a significant part of compilation time. Disabling it would be very detrimental for little gain.
Here is an older article that compares compilation times between multiple different libraries, where one of them does spend a significant (and majority of the) time borrow checking: https://wiki.alopex.li/WhereRustcSpendsItsTime