Fortunately there are plenty of tools to handle this, and at a hobby level not particularly resource intensive. Something like this is simple and reliable in many cases: https://github.com/berstend/puppeteer-extra/tree/master/pack...
Not really. In a lot of cases websites use JavaScript to call some API along with some on the fly generated token to prevent abuse.
As long as that token isn't captcha you can reverse engineer the site to do scraping without javascript and that is so much faster than browser based scraping.
Its just a cat and mouse game. After few year I think hardware attention etc will come to play which can mitigate bot issue somewhat.