Hacker News Reader

Comment by daenz | Hacker News Reader

> It's an arms race, but the problem is, only one side can win.

I don't disagree, but although this specific problem can be automated, the arms race will continue as newer problems (requiring uniquely human intelligence) are presented. We can keep moving the bar until the singularity.

For example, a captcha: "Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed. To solve this, AI would have to have a mastery of language, a mastery of object identification, and a mastery of the metadata of those objects as they apply to human safety.

View on HN

That runs into the scalability problem that I mentioned--someone (say, a Mechanical Turk user) can solve that problem once, and put it in a database. To avoid this, there needs to be a way to generate them cheaply, so you pretty much need a computer that can solve the problem in the first place.

Basically, my point is that there's no such thing as "uniquely human intelligence". There's lots of problems that fall into that category now, but with the continuing rise of computational power, it's getting harder and harder to find new problems of this sort without excluding a lot of real people.

So that begins to look a bit like the RSA problem, doesn't it? Find a problem which can only be solved efficiently given some "secret" knowledge – where, instead of that knowledge (the prime factors) being secret, the knowledge is something it's easy for a computer to use but hard for it to recognize.

I'm probably overgeneralizing, but it's an interesting framing.

It seems more likely that there's no such thing as "computer-identifiable human intelligence". Human intelligence is easily discerned by other humans.

>To avoid this, there needs to be a way to generate them cheaply, so you pretty much need a computer that can solve the problem in the first place.

I don't see how that follows. Many captchas are just text with a confusing grid overlayed - that's incredibly trivial to produce, and producing one tells you next to nothing about how to solve them. Or if that's too easy, throw in some random distortion - still easy to produce, even harder to read.

But one requirement of a captcha system is that it be easy for a computer to generate. The system you describe seems to require that humans build up a list of things with obvious-to-human answers, which doesn't scale, especially when they have to be designed under the assumption that "the enemy knows the system" (i.e. a worker on some "library of common knowledge" project could defect).

But if you could have an AI generate those problems, it would be capable of solving them too!

Current captcha systems exploit the one-way nature of problems, mainly "character distortion". This is what permits easy computer-generation but not computer-solution, and does not seem to be present in the class of problems you describe: it's just as easy to go from "noun -> adjective" as "adjective -> noun".

"Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed.

My sense is that IBM's Watson could tackle this particular question. Also interesting to note that the question introduces external considerations, such as whether or not there's someone dangerous near the house, bed, park, etc. A human respondent will have to ignore the matter of whether there's additional context and second-guess the captcha author to get the problem right -- presumably the most unsafe environment is the volcano.

Of course the simple way around

"Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed.

if you're a spammer is to choose randomly and make four times as many attempts. Which is why catchpas make you type six or more letters giving millions of combinations so you can't do it randomly. To get up to a similar resistance to random attempts with 'pick the unsafe environment' type questions you'd need of the order or eight of them (giving 4^8 or approx 65k combinations). Would you as a user want to fill in 8 of those things?

The key is that you are only presented with one captcha per request. But if you fail too many, you get labelled a spammer.

Like Deep Blue, Watson represented a major advance in the state of weak AI. However at its core, it's still just a text-based statistical search engine. It has no ability to "learn" new abilities. Its algorithms will use previous answers to adjust ranking of answers, but once a category is finished, that data is lost. For new data, its database needs to be re-indexed. For new behavior like recognizing pictures or playing a game, it needs to be programmed.

Watson played a simplified version of Jeopardy with no video or audio questions. This was a key compromise to allow a text based system to compete with humans. The AI problem of recognizing pictures is practically unsolved. It took Google's best researchers, a massive database of cat pictures, and a supercomputer just to train a computer to recognize when a cat was in a photo. So your sense of Watson's present abilities is a little skewed and unrealistic.

Yes its interesting; you can easily imagine that the following trends are detected:

* "home and safe" "safe as houses" etc

* "safe in bed" "safely tucked up in bed" etc

* "volcano" ... no clear data. Certainly dangerous being a volcanologist, but that'd be great AI googlefu

* "pensioner attacked in park" .. "our parks are not safe" etc... Hmm, Parks are the most dangerous.

Because frequency of local newspaper reports is a bad measure for volcanos I guess.

While probably not 99% reliability, I think google software could often solve this one too: Reverse image search could provide an image description, which combined with number of search queries for eg 'unsafe $image_description' to give a ranking. Google can already mostly handle synonyms; it does not need a mastery of the area as it can just use correlations and usage patterns.

Also, being solvable for humans and not computers is not the only criteria for a captcha. For example it must be quick and relatively easy for humans, and provide a definitive consistent answer. The brilliance of recaptcha is that it was making the work that humans excelled at actually useful beyond just 'proof of humanity'.

Or, you could use xkcd's idea: http://xkcd.com/810

A bot would solve this one with a random selection 25% of the time. You would need multiple puzzles to make this a useful solution.

The odd thing with a question like that is how "most dangerous" is determined. Logically the volcano is the most unsafe environment, and people do die on them, but statistically far more dangerous, at least by death count, is the bed.

This is the kind of problem that would still cause issues for some people, especially if you had to scale them to produce identical captcha's relatively rarely (if the same one shows up too many times it can easily be hard-coded, requiring the most basic detection to solve it reliably) in that as you come up with more of these types of questions the expected answer gets rapidly more nuanced.

So it's a captcha that keeps out computers and pedants on HN. Win/win, in my opinion.

Dangerous does not necessarily mean fatal. The posed question specifically states environment, and the bed as an environment is typically a flat, soft, blunt, rectangular prism.

Whereas volcanoes, and more specifically, active volcanoes can be unpredicable and spurt out hot lava and ash.

The question is worded fine and your attempt to poke holes into this is invalid.

Even if a question is worded well, it can always be interpreted incorrectly. The question should not be blamed for a person's inability to process information.

At the same time, though, a human would have to have familiarity with all of those objects in the context of their environment. A house doesn't look the same to everyone, nor does a park, bed, etc. People's life experiences can also affect their responses.

The problem we would then experience is unintentional prejudice and Western-mindedness in our "are you human" queries, similar to a problem that SATs and other standardized testing methods have been criticized for.

>To solve this, AI would have to have a mastery of language, a mastery of object identification, and a mastery of the metadata of those objects as they apply to human safety.

Actually for a captcha to work, you have to have a problem that is easy for an AI to pose and verify the solution to, but hard for AI to solve. This could potentially rule out some classes of working turing tests.

Reply on news.ycombinator.com