Is there some way to declare, "I am a legitimate academic user", something akin to 'TSA Pre' status?
"Sure, register for & use the site's API," you'll say. What if they don't have one?
"Sure, just don't slam the server with too many requests in a short time," you'll say. But if they're rejecting you just because they detect you're headless, etc...?