If you're getting a lot of crawler traffic, your site probably has information a lot of people find useful, so you should consider finding a way to monetize it.

Otherwise, your best bet (hardest to get around in my experience) is monitoring for actual user I/O. Like if someone starts typing in an input field, real humans have to click on it beforehand, and most bots won't.

Or if a user clicks next-page without the selector being visible or without scrolling the page at all. Not natural behavior.

Think like a human.