That's the thing I don't understand about the current situation.
That's the thing I don't understand about the current situation.
2. However, more to the point, what's the end game there? If there are 10,000 outlets out there; and arbitrary number, say 100 of them block scrapes; now the other 9,900 will show up in Google/Facebook search and shares, and those who implemented scraping protection will quietly fade into non-existence.
It's a damned-if-you-do:damned-if-you-don't choice of modern media consumption model. I assume that most outlets have decided that the exposure, trickle of users and monetization they DO get is superior to the influx of users drying out. It's consequence of the oft-quoted platitude that these days, for most people Google and Facebook ARE "The Internet" :-/
Not especially, since internet browsers make GET requests the same as scrapers. There are techniques around blocking user agents, rate limiting, etc. but that's an arms race against the scrapers.
The tricky part about blocking Google is that if you do, your site won't show up in Google search results. :)
They clearly want the benefit of being scraped (the traffic) without the cost (the loss of traffic from people who don't click past the headline to the article).