Hacker News Reader

> An agent has nothing to process if there's no text on the page

all of the major labs' flagship models are multimodal and the post is specifically about processing the non-text parts of the page

View on HN

A good point. I was unclear. I meant it in the sense that if the JS doesn't run fully successfully there won't be text, or images, or even source mark-up. Just some lists of remote JS CDN in source if anything. It'll mostly just be blank. Nothing for the multi-modal to do either.

Why would the Javascript not run? Crawlers have been executing scripts for more than a decade now.

Because what javascript is changes rapidly and they always have to change to keep up and we all agree that that amount of work is difficult. So, parts of JS fail and don't run as expected over time even with constant mantainence.

the models are clicking around in real browsers

This argument makes no sense, in that case the page would be broken for regular users all the same. Scrapers don't need to use some inferior browser and then pick up the pieces, they can literally use the same browser all the regular users do.

I think you'll find that people with complex screen reader setups don't chase the bleeding edge quite as much, in general.

Reply on news.ycombinator.com