The poorly written bots can be detected looking at TCP header information such as MSS, window size, sequence numbers, TTL, packet length. [1] These are some of the things that CDN's like Cloudflare look at. This probably won't get you to your desired 80% but might get you half the way there.
Another method I have used to squash a handful of bots is to require HTTP/2.0. Many of the older bots only speak HTTP/1.1 or 1.0. The modern/common browsers can speak HTTP/2.0. This method may break some API CLI tools if your site has an API gateway. This can be worked around by putting the API gateway on its own load balancer or web servers. I should add that Google's crawler bots can not yet speak HTTP/2.0 so exclude them from this restriction.
Another method is to implement strict-SNI on your load balancers. Some bots can't deal with this yet and will end up on your default dummy site or just get an error depending on how one implements this.