Mocks, especially spies, couple your tests to the implementation of the system under test. This leads to a very common code smell in tests: Fragile Test. The problem is that instead of being concerned only with behaviour, the test are knowing something about the implementation details of the SUT (dependencies are an implementation detail that consumers ought to be unaware of).

One of the major benefits of automated tests is that they can provide a safety net that allows you to refactor with less risk of breaking behaviour. If tests get in your way of refactoring, because changing the implementation details of your SUT cause your tests to start failing, then the tests have failed at being a tool to facilitate refactoring.

Mocks and stubs provide lower test fidelity than the real thing. Also makes your tests more brittle as you need to update them whenever the real implementation changes.