Hacker News Reader

Comment by pests | Hacker News Reader

What’s wrong with it downloading documents when the user asks it to? My browser also downloads whole documents and sometimes even prefetches documents I haven’t even clicked on yet. Toss in a adblocker or reader mode and my browser also strips all the ads.

Why is it okay for me to ask my browser to do this but I can’t ask my LLM to do the same?

View on HN

When Google sends people to a review website, 30% of users might have an adblocker, but 70% don't. And even those with adblockers might click an affiliate link if they found the review particularly helpful.

When ChatGPT reads a review website, though? Zero ad clicks, zero affiliate links.

So if enough people used adblockers that would make them bad too? It’s just an issue of numbers?

Brave blocks ads by default. Tools like Pocket and reader mode disables ads.

Why is it okay for some user agents but not others?

There’s nothing wrong with downloading documents. I do this in my personal search app. But if you are hammering the site that wants you to calm down, or bypass robots.txt, that’s wrong.

robots.txt is for bots and I am not one though. As a user I can access anything regardless of it being blocked to bots. There are other mechanisms like status codes to rate limit or authenticate if that is an issue.

I'm talking about perplexity's behavior. Perhaps there's a point of contention on perplexity downloading a document on a person's behalf. I view this as if there is a service running that does it for multiple people, then it's a bot.

Perplexity makes requests on behalf of its users. I would argue that’s only illegitimate if the combined volume of the requests exceeds what the users would do by an order of magnitude or two. Maybe that’s what’s happening.

But “for multiple people” isn’t an argument IMO, since each of those people could run a separate service doing the same. Using the same service, on the contrary, provides an opportunity to reduce the request volume by caching.

Reply on news.ycombinator.com