Hacker News Reader

Comment by dimmke | Hacker News Reader

I recently conducted an experiment - I removed client side CAPTCHAs from a form that had reCAPTCHA V2 and a ton of spam was getting through and instead sent the content to Akismet for scanning. It cut the spam getting through to 0.

It made me think, are client side CAPTCHAs really worth it? They add so much friction (and page weight - reCAPTCHA v3 adds several hundreds KBs) to the experience (especially when you have to solve puzzles or identify objects) and are gamed heavily. I know these get used for more than form submissions, to stop bot sign ups etc…

I feel like it’d be just as/more effective to use other heuristics on the backend: IP Address, blacklisting certain email domains, requiring email validation or phone validation, scanning logs, analyzing content submitted through forms

View on HN

Then you'd just give visitors of your websites no recourse and no information whatsoever on how to fix the problem. The benefit of client-side CAPTCHA is that humans at least can pass it and fix the problem even if something they don't control (such as their IP address having bad reputation due to shitty ISP) is causing problems.

As a website operator it's easy to look at the spam that is getting through and be happy that's it's zero. But do you get any idea how many actual humans that you have incorrectly rejected? You don't have that data and it's really easy to screw up there.

Of course if your website is small nobody cares. If you are bigger like Stripe you simply get bad publicity on HN. People on HN love to hate on mysterious bans and blocks just because they do something slightly unusual and your backend-only analysis flags them as suspicious.

Abuse fighting is hard.

And just to be clear, it doesn't need to be either captchas or doing heuristic abuse detection on the backend. In the ideal case you're making a decision in the backend using all these heuristics and signals, but the outcome is not binary. Instead the outcome of the abuse detection logic is to choose from a range of options like blocking the request entirely, allowing it through, using a captcha, or if you're sophisticated enough doing other kinds of challenges.

But proof of work has basically no value as am abuse challenge even in this kind of setup, the economics just can't work.

>Then you'd just give visitors of your websites no recourse and no information whatsoever on how to fix the problem.

This is a weird assumption. What's preventing a backend system from saying "Hey, we think you're a bot. Here's an alternative way to contact us."

You obviously don't want to give away enough to help bot developers get through your system, but that's not the same as no resource and no information.

>But do you get any idea how many actual humans that you have incorrectly rejected?

Yes - like I said in my other comment, this new system actually logs all submissions. It just puts the ones it identifies as spam into a separate folder. Akismet also has the ability to mark things as false positives or false negatives.

I think that automated form submissions are very context specific. So, the example I wrote about is for a marketing site, and it's a business that primarily targets other businesses. Most of the spam it gets is for scummy SAAS software, SEO optimization, etc...

But my personal website has a very simple subscribe by email form. There were definitely a few spam submissions - someone just blasting out an email address and signing it up to whatever form would accept it. When I implemented double opt in - gone entirely.

My larger point was that as an industry, we seem to have just capitulated to client side CAPTCHAs. And it sucks. It's one of the many shitty things about the modern web. But I think it's become just an assumption that it's needed, and we haven't reexamined that assumption in a while.

I think it'd almost be better for there to be something could spin up in a container that has a base machine learning model, but can "learn" as you manually indicate messages etc... and then you can also choose a threshold based off your comfort level.

> This is a weird assumption. What's preventing a backend system from saying "Hey, we think you're a bot. Here's an alternative way to contact us."

Not a weird assumption, but a necessary assumption based on considerations of scale.

A small-scale website that doesn't receive too much spam attempt can manually classify spam by human agents. A medium-scale website can have CAPTCHA to let through some visitors and the rest goes to human verification. You appear to be in this bucket. When the scale is huge, no other alternative way to contact exists. CAPTCHA becomes your only tool.

In other words, CAPTCHA is only necessary because of scale; what do you think the first A stands for? But because of scale, alternate ways stop working.

>When the scale is huge, no other alternative way to contact exists

1. This still doesn't preclude giving a blocked user recourse or information. Like how a streaming website will say "Hey, you're using a VPN. We don't allow that" - the user's recourse is to turn off the VPN, or find a new VPN that their service won't detect.

2. The case you're outlining is different from the scenario that most users are presented with a CAPTCHA. I encounter it when I am using a VPN and Googling something with Incognito mode. That means Google has already applied some heuristics and thinks that chances are higher than normal that I'm a bot (not logged in, no cookies allowed, masking IP address) before presenting the challenge. In those cases, you're probably correct that presenting a CAPTCHA is a reasonable option. I just think it's weird to have CAPTCHA be the default/first line in many cases. Especially with the focus on things like converting users.

> Like how a streaming website will say "Hey, you're using a VPN. We don't allow that" - the user's recourse is to turn off the VPN, or find a new VPN that their service won't detect.

No, the user's recourse is to stop using the streaming website and go back to piracy instead.

Any speedbump to UX is a lost customer. You can not and should not assume that users are going to jump through hoops, because the overwhelming majority will not.

I mean, the vast majority of people will not "go back" to piracy. Piracy isn't an option that's on the table for them. But you're missing the point.

>Any speedbump to UX is a lost customer. You can not and should not assume that users are going to jump through hoops

So... CAPTCHA isn't a hoop? Both scenarios are hoops.

> What's preventing a backend system from saying "Hey, we think you're a bot. Here's an alternative way to contact us."

In what way? If I got flagged and had to take additional steps to remedy a form submission I would probably just never go back to the site. The only way this could work is by identifying the issue in real-time and then sending a CAPTCHA to be completed by the user client-side while they're still handling the form.

> to remedy a form submission

The correct way to deal with an error in general is to return the form as filled by the user on the error page. Sadly so many SPAs just ignore this, that I usually manually copy before submitting any meaningfully sized text.

> I would probably just never go back to the site

Of course you would need to measure and reasonably reduce false positives. But in case you are serious about getting users to report them, an effective solution I've found with minimal friction is to fully use the mailto protocol scheme. Online shops can be static sites up to a scale, by adding product IDs and quantities to the mailto body, and having the customer order via e-mail.

> The only way this could work

Depending on your target audience, a CAPTCHA might not be possible.

>an effective solution I've found with minimal friction is to fully use the mailto protocol scheme.

This is another assumption that I am not so sure is valid anymore: that there are hordes of bots out there scraping every email they can find.

My personal website has had a raw mailto "Contact" button for several years now. Earlier this year I changed that email to an address I only use for the website (it's an alias) just as a way of tracking what comes through, and I have not received a single spam email to that address. Maybe I am tempting fate by putting that out there, and some asshole is going to try to ruin it for me. But it's my experience.

I'm more likely to get an email from a recruiter who has used a tool to scrape my email off of Github (though I've made that significantly harder plus nobody wants to hire programmers anymore the ultimate spam control!) than an email from them having clicked through to my website and using the Contact button.

I've gotten several real people sending me legitimate emails through it though. Sometimes they read something I post here, or they find a post from Google. Or it's someone I haven't talked to in a long time and they don't have a current email for me but they are able to reach me via my website.

Here's some cold emails I've been sent over the past 1-2 years: https://i.imgur.com/rMfOqb2.png

My website isn't big, but it is fully indexed by Google and other search engines and has backlinks. If the internet was really this dark forest where there are masses of bots out there ingesting every email they can find, surely I'd have gotten something by now.

I don't research this stuff, I can only share my anecdotal experiences. But it makes you wonder, right?

I think the idea here is that 1% of users with the shitty ISP is going to have a much worse experience than anyone was having with the captcha. This is super context-dependent, but being told I need to "contact an administrator" when I submit a form on a website is a good way to make me log out and look for alternatives to whatever service I'm using.

To me, the question is this: would you rather give 100% of your users a kinda shitty experience, or 99% of your users a normal experience and 1% a nightmarish awful shitty experience. The answer probably depends on use case.

Client side captchas are obfuscated code, so the bar of “the user can debug the problem and fix it themselves” is pretty high.

Also, reCaptcha definitely engages in both hell-banning and allows incorrect answers to pass the test. I assume the logic for those things is mostly server side.

I added this trivial honeypot field to a site’s register-interest form in late 2021, and it has been very effective at culling spam: I had been getting an average of around one spam message a day, but after adding it it took a year and a half for one to get through, and no others have got through.

  <style>.pot{display:none}</style>
  <div class="f pot">
      <label for=username><b>If you are human, leave this field blank:</b> <em>(required)</em></label>
      <input name=username id=username>
  </div>

The whole field is hidden with CSS; the “if human, leave blank” instruction is thus only used for text browsers, but I still prefer to have it.

As a text browser user I thank you for adding that label.

I wonder if this is filled out automatically by browsers like Chrome when doing auto fill...

I would be extremely surprised if it filled it: `display: none` makes the field not be rendered, and autofill should only fill stuff the user could fill.

Akismet is a paid service and their apis are tailored for comments. An advantage with comments is that you can just mark as spam if some contents have dubious links or keywords.

An issue you have in many forms (e.g.: login form) is that there is limited data to decide if it's a real user or a bot.

Agreed on all points. That's why I said in my original comment: "I know these get used for more than [contact] form submissions, to stop bot sign ups etc…"

I picked up Akismet because it's been around forever, and while it is paid, it is very cheap for my use case.

This is a bit of an aside, but I feel like Automattic is sitting on several companies/products and not doing a whole lot with them.

Akismet could be expanded into a more fully featured server side spam detection SAAS with a flexible API etc...

Gravatar could be expanded into something like OpenID.

Just seems like a waste to me.

Did you also look at the false positives, e.g., how many non-spam content was filtered by Akismet?

Of course. It would be bonkers not to. It just doesn't send a notification if the submission is flagged as spam and puts it in a separate folder. So I have the ability to look at every submission.

I put the system into effect on August 1st. There have not been any false positives. There was even a submission to the form that was clearly a B2B sales pitch, but because it was an actual person submitting the form and not an automated system it went into the "real" entries list (I think this is reasonable. Any business is going to have to field B2B sales solicitations)

I put together a few rows in a spreadsheet of legitimate submissions (with info blocked out): https://imgur.com/a/stxja1Z

Here's an example of one flagged as spam by Akismet that was submitted about an hour ago: https://imgur.com/a/PmN3t80

Overall, removing reCAPTCHA has increased the total amount of submissions to the form, but the amount of submissions actually being seen by a real person who then has to waste time reading it, identifying that it's spam and discarding it has dropped to 0.

I wonder if it would be a decent approach to scale CAPTCHA tests based on how likely an LLM thinks a post is spam.

People who WRITE LIKE THIS might just end up ACCIDENTALLY BEING FILTERED and using click as an imperative is a complete red flag.

I think you could probably filter a number of these out with just regular NLP approaches/models even.

Sounds like you should write an article about this! I've never heard of Akismet so I'd be curious to see more info on your findings.

CAPTCHAs today are horrible. I don't even bother with most webpages that require them at this point. In a similar vain, I don't bother with sites that send me in a never-ending Cloudflare loop just because I lock down my browser to limit what sites can and cannot do. It's particularly tyrannical when I am either sent in a loop or asked to complete a captcha for a non-interactive page.

Some spam is really low effort. Even a non obfuscated text image (e.g. something that can be read by teseract out of the box) still stops a surprising amount of spammers.

The answer is something like "yes, and..." because reCAPTCHA already decides whether and how to challenge the user based on its own internal risk score.

But if a server side only solution seems to work fine, why add a client side element?

I ran into this because I was doing some freelance work on a website that had worked its ass off to cut loading size as small as possible. For reCaptcha to develop its risk score, you have to load it far in advance of a form - you can't lazy load it, they specifically say not to: https://developers.google.com/recaptcha/docs/loading.

It also spawned its own service worker on top of adding like 300kb of page weight. The API documentation is garbage, you have to fuck around with Google Cloud to get API keys now too which is confusing. It also pollutes the global scope of the page. It's all around terrible to work with.

I think that's one of the benefits of using things like Auth0.

They have thousands of companies so their heuristics are really good.

Reply on news.ycombinator.com