Hacker News Reader

Comment by BorisMelnik | Hacker News Reader

my prediction: ai agents will have their day, but not on this version of the web. websites will have a separate xml/json/yaml version that agents can peruse without all the ads/js/etc

View on HN

Isn't that just API? We had them. Reddit/Zillow/Wells Fargo/etc restricted theirs due to business reasons, not technical ones.

My agents can just take that feed and render it into a custom front end just for me too. Why would I browse the adversarial, bloated, ad infested version?

Why would that happen if the website owners are explicitly trying to prevent access? This isn't a technical issue, the agents are perfectly capable of working with the existing websites regardless of ads and javascript. The semantic web and machine readable representations was a 90s thing, agents don't need that really.

the js/yaml version would require a license key that connects to a business arrangement between the website and the owner of the crawler. the plain HTML version would adversarially block them (or even identify them, where they could be sued...once we have legal precedents for this stuff).

no you are def right the agents are def capable and that is the problem, so capable we are all doing this, we are all making our own scrapers, as are big corps/small corps and it is adding up so now we are seeing managed challenges everywhere and webmasters/sysadmins going to great lengths to block any bots.

And finally, we will have the future SOAP and XML schemas promised us all those years ago!

Websites cost money. Websites won't cater to bots the same way they don't cater to the scrapers of old. Some bot companies may be charged extra to integrate their bots in some websites, though!

I agree. I’ve been saying for years that marketing teams need to switch to include marketing to LLMs.

I have (currently free but maybe paid in the future) accounts for side projects with businesses that Claude chose for me. (I reviewed the suggestion, but didn’t see a problem or reason to change it.) Why did Claude choose those providers? No idea, but if it’s through subtle marketing to Claude, then well done to them.

AI agents have already colonised the human internet. No one is gonna make them switch over. It’s the humans that are gonna be displaced.

Without the ads? The pessimist in me thinks with all the ads, prompt injections, and long-form content marketing included. Already you can see Google's AI results rehashing marketing slop in the answers about how amazing one product or another is.

Wasn't there some restaurant review site that gets accused of doing pay-to-play hiding of negative reviews and promoting positive ones that calls up businesses and basically extorts them into their services? They have all the incentive in the world now to manipulate the results shown to AI agents. You won't even see the manipulation directly yourself when there's an AI sitting between you and the source acting like an obfuscation layer.

hm. for me, AI agents effectively already have their day unless there is a retrenchment. i essentially do not interact with the web (outside of a few social media sites) anymore

>i essentially do not interact with the web

Grim. No point in writing blog posts or publishing anything for free if that's the future.

There IS actually. In my case I'll look at a site if an agent says it's good!

Yeah the web is done, youtube is probably the only place worth publishing to now.

if the reason to do so was the ad revenue, then i definitely agree. but in terms of me happening on to your ideas, now is perhaps actually better than any time in the past. i also do pay for writing à la substack

Reply on news.ycombinator.com