Hacker News Reader

I think some crawlers run JS, because a lot of the web simply won't work without JS to initialise the page state these days.

You can use captcha or similar, one workaround I've seen has a submit that is hidden so never clicked by real people then a visible submit that sets a hidden input and clicks the other one which requires the hidden input... not foolproof but avoids some accidents.

View on HN

A crawler that follows links found in emails and sends POST requests / submits forms will cause so much havoc. It could buy things, validate account sign-ups, delete data, etc. I have a hard time believing the answer isn't to ask users to switch to a normal email provider.

I know web search at least runs JS to get better results. Not sure about email pre-fetch but I assume they do. I don't think crawlers click buttons though unless they are malicious so it's probably fine for unique email links.

Reply on news.ycombinator.com