They also scrape and cache sites on behalf of the user too. I understand why media sites are upset about their lack of control over their content, but that's just how the internet works. Sites and services proxy requests for their users all the time, and since this is live scraping directly on behalf of the user from an affirmative user action, it's not really materially different from a proxy.

You could simply shift the site requests to an end user machine instead and then have that data be sent to your server for processing, so all this seems like a technicality of what machine is doing which work. It's very different from an indexer like a search engine that does batched crawling.

I don't understand how these archive sites aren't shut down yet. Is it because only a small sliver of internet users know about them?