Hacker News Reader

Comment by ddebernardy | Hacker News Reader

Personally, I'd argue the opposite: AI today is super impressive compared to what it was 15 or 30 years ago.

There used to be a time when one could rightfully dismiss new cool things AI with "meh, wake me up when an AI beats a professional Go player on a 19x19 board". Because doing so requires an AI that displays intuition and use of heuristics (Go moves tend to have both local/tactical and global/strategic consequences on the board) rather than one that merely brute forces its way through a decision tree.

The fact that one actually did recently is nothing short of game-changing in my view. It matters little that an AI doesn't demonstrably understand meaning or fully grasp intent. Plus, when one does, we'll arguably have (or not be too far from) a strong AI that, I would gather, won't be too impressed with our own human-level thinking. :-)

View on HN

To me impressive would be if an AI could be told a story, such as The 3 little pigs and then be able to reason about it speculatively, e.g. answering questions such as "Why do you think this character did this and that?". This would should that the machine has developed an intelligence. So far, all AI can do is pattern recognition, which can be very useful for many tasks, but in my opinion it shouldn't be called "intelligence" because it isn't. Far from it.

I agree about the story. Text generation is already well within our capabilities, but it would be much more "intelligent" if the algorithm understood concepts of plot and characters to come up with a coherent piece of writing. There are already algorithms for summarising sporting and news events, but that's pretty formulaic, not creative like writing a story.

I want to be careful here; yes, semi-coherent text generation is possible under the current state of the art, but there is no evidence right now that it 'on the right track', as it were, to solving the harder problem of binding semantics and language that you allude to; it may be the case that our current techniques for text generation, inverses of our current techniques for text 'understanding' (and I use the term loosely) are just sufficiently powerful to pick up whatever signal is available from the surface-forms of language, but the information is too scarce, the entropy too high for that next big step of 'why'

This is actually one of the things AI is currently being developed for, and may not be that far off. Again, it won't be reasoning as we understand it, more a set of state tables that allow inferences.

But people are definitely trying to develop AI that can take a description of an event, and break it down into legal concepts such as "who was at fault here?"

I agree that that would be impressive, but beating a human being at Go is also very, very impressive, and had been considered a high benchmark for many years. I can't help but think that after a machine can do basic story analysis, there will still be people saying that this is just a trick, that doesn't show any "intelligent" analyses of the story.

AI has a long way to go, but I don't think we should dismiss intelligence as an either/or binary, and call things that are not "intelligence" mere "pattern recognition." Strategy games are widely accepted across cultures as a mark of intelligence when humans do them; you'd be very comfortable saying a human chess or Go grandmaster was intelligent. There was a long time when people were dismissing basic AIs as mere toys by saying they'll never beat a human at Go. There's a little goalpost-moving any time someone responds to an AI breakthrough by saying "You know what would be really impressive..."

The Go victory is definitely impressive, for what it was, but I think this is a pretty unfair comparison; it's an entirely different class of problem, and a strictly less difficult one at that (at least given current knowledge of the mind).

It seems likely to me that strategy games are accepted as an intelligence gauge partly because they're so hard for humans to do; thinking so many plys ahead by exploring and pruning across the game tree seems really hard for humans. For some reason, Natural Language and its semantic binding (I think the notion that these two things are often treated separately is part of a lot of the bigger deltas between expectation and reality when it comes to 'AI'. Honestly, they're probably the same thing anyway) doesn't seem really hard for us.

That doesn't strictly imply that 'strategy games' are strictly harder than 'natural language understanding'; in fact, as far as research goes, strategy games are easy to tackle, because we know all the rules; we designed them! And often, perfect information is available to all agents involved. It's about as black and white as problems get. Sure you can't brute-force Go before the heat death of the universe, but it still theoretically -can- be brute forced. The rules of the game (known and accepted by all parties) encode the algorithm, and it's just a matter of reverse engineering it step-by-step for a given situation. What a brain is doing when it speaks or listens or thinks is currently a much bigger, much more primitive mystery. As far as we know right now, you can't just brute force sentence permutations until you hit a correct answer to a question like the one posed by the parent.

So I don't call this goal-post moving (though acknowledge that the Vanishing Problem is systemic in AI Research and Pop Science); instead, I think this is more akin to the work in say, separating problems into classes of computational complexity. Finding the upper limits of things can be very helpful when we're dealing with such an old, deep, controversial problem.

For sure: as much as Go really was widely-talked about as an AI goal, real language processing has been the goal since, well, Turing. Even more importantly, it remains to be seen how far the current neural network approach extends, and it could still be only successful at certain tasks, with a bigger paradigm shift needed to crack language and so on. But I don't think it's worthwhile to make a hard, fundamental distinction between game-playing and language, such that one requires "real" intelligence and the other is something else.

After all, we possess a single machine that can both play Go and use language, and do far more things besides, and it really does look like the brain is re-using a lot of its structures for these different problems. You can theoretically solve Go, and you can't solve language, but the brain doesn't try to solve either (and neither does Deepmind); there's at least some hope that one paradigm can be applied to both these problems, and the reason we originally turned to neural networks was because of their (admittedly highly superficial) resemblance to our one positive case of intelligence.

So I do take issue with the idea that language understanding is "strictly" harder than board games; that's an artifact of the paradigm used. The idea that the number of rules and exceptions to rules is the 'objective' mark of a task's difficulty only applies to a certain kind of processing agent, and our one example of intelligence clearly uses a different one. For most of the past ten millennia, the only information processing agents on the planet had far more ease communicating with natural language than crushing at board games; ask some now to either write an essay or beat Lee Sedol 4-1, and they'll tell you which is easier. The fact that board game mastery is easier for our AI attempts than basic speech isn't a sign that they've secretly been easier this whole time; for a certain kind of processing agent, communicating in a poorly-defined space has been much easier than mastering a clearly-defined one, and if our AIs don't match that pattern, it's only another reason to wonder if we're on a different path.

The 'kind of processing agent' emphasis is a very good one, and I think worth bringing up in every one of these conversations.

That said, I want to niggle a little bit on some of these points

There certainly does seem to be component re-use in certain parts of the brain between seemingly disparate tasks. The left brain/right brain divide has been largely debunked, and I'm not entirely sure what the current state of the art is, but I'm not sure I'd go so far as to say that that re-use in this particular case (if it exists, I don't know) is anything other than a 'best approximation' (and maybe not even best) with the available hardware. Like using a fork to eat soup. Sure you can kinda do it, but you're always gonna be thinner than the fellow with the spoon.

That suggests to me then that using games as a metric of intelligence is actually a pretty poor way of getting at the deeper issue; there are too many other variables beyond "how much (general/amalgamated) smarts have you got?". It might be semi-convenient when you have cause to believe that the two models you're testing have largely similar structure (i.e., two humans), but it gets pretty noisy when you've got a less than solid idea how similar the two models are. Because there is an obvious, perfect solution to the problem, you end up with a kind of algorithmic continuum, with the system you're trying to model on one end and perfect brute force on the other. And you don't know how far along the continuum your model is; it might be pretty close to the system (the brain), in which case, hey! Progress. But it might be closer to the perfect solution, and thus closer to dedicated hardware for the problem at hand, instead of adapted hardware doing its best. It might have a spork, which makes it rather less worthy of admiration when it comes to soup-eating. Especially when you see it struggle with the steak.

So, in short; I think it actually -might- be a sign that strategy has secretly been easier this whole time. I'm not even sure it's that much of a secret. At least when it comes to the particular processing agents in question, the ones capable of general intelligence as we understand it. If you want to get into philosophical conversations about Hyperintelligent Shades of Blue, well, that's another story.

It is super-impressive, but peoples expectations are also massively higher because more and more people have become accustomed to things that were "crazy" 15-30 years ago.

Even so, to paraphrase William Gibson, the future is not very well distributed.

On one hand people expect miracles and are not very impressed with systems that are incredibly impressive. On the other hand I've been showing some a side-project that mostly just uses a basic Bayesian model that people get impressed by seemingly because it's in an area where they are used to poor results (I'm using it for a very basic text categorisation problem).

People in general appear to be very bad at extrapolating possibilities and mostly judge these things based on what they're used to rather than any assessment of what "ought" to be possible.

This is also true in research in the sense that the moment a problem is solved that previously felt unattainable, it gets somewhat dismissed as "weak AI". The goalposts are constantly moved and the progress to date is underestimated.

My friend did a simple python script that asked "give me a few words, with one being in some ways different than the others", and the script returned the word that was different.

So, for "dog, snake, sheep, tree, owl" it would respond with "tree". For "blue, green, sky, red, black" it would respond with "sky".

It worked by posting a query to google with all the words except the first one, then all the words except the second, etc. And then it compressed the results, and the word that resulted with the smallest compressed result when removed - was the odd one. Very nice use of Kolmogorov complexity estimation.

It still blows people' minds few years later, when I describe what it could do, but then I explain how it does it - and they react with "meh, a simple trick, not an AI".

That's how people react to AI in general - it's only AI if you don't understand it.

I think the point on this story is a bit different than you are interpreting it.

What you're basially describing is the same issue that magic tricks have, where it's fantastic until you know the secret which is often simple.

The difference is that with magic, no (good) magician is claiming they actually have magical powers, and that's why some people get really skeptical and dismissive of AI claims. (Not to say you or your friend are claiming you had strong AI, but more to the modern dsicussion of AI in general)

The problem really is that there are a lot of people commenting on AI who exist in the "know enough to be dangerous" category of technology. They don't fully understand things about computers or programming, but they know enough to sound like they do. This, combined with the fact that extraordinary claims and articles about AI is great publicity for businesses, there is a tendency towards exaggeration when it comes to the idea of AI. For example, I personally wouldn't call what Google, et. al., do as "AI", but as a subset of the field, focusing a lot on speech analysis. This isn't nearly as evocative as saying "AI Research", but it's much more accurate and I think would prompt more intelligent commentary and questions.

When I read your story, I thought it was cool and was curious how you did it; I didn't ascribe AI attributes to it, however, and still found the technique clever as a result. When I hear a claim about "AI" from news currently, I find myself having to distinguish what they're actually talking about first. When there is an inevitable article about AI and people claim that AGI is right around the corner, that is when I get skeptical, and when I think that those who do know a bit about computing get very skeptical and dismissive, since they expected a bit more than more stuff related to just keyword searches. The simple fact is that, like magic tricks, our imagination for AI right now is a lot more impressive than the reality behind current AI research, and a lot of what is presented as AI is more focusing on a specific subset of AI with an impressive front-end to it.

This isn't to dimish the work that the researchers are doing, but clarity of language would be preferable to just the umbrella "AI" category.

I wonder if, as neuroscience research improves, intelligence itself will seem less "magic." As you say, good AI researchers shouldn't claim to have magical powers--as far as we can tell, only these lumps of meat in our head are magical!

I honestly hope so. Not sarcastic or demeaning, I just think that neuroscience is what is necessary for the mythical "AGI", since we don't have a strong notion. I don't believe for a second that it's unknowable, but right now I don't think we know.

I just published technology that some researchers called an 'AI pitch' about 'magic AI does everything' (https://news.ycombinator.com/item?id=12167097#12171264), even while they graciously pointed out some of its optimizations that add up to 'really good tech'. I appreciate all of the constructive and critical feedback, and I'm trying to bring it all together.

Probably the greatest conflict I feel in communicating about such R&D is the challenge of resolving two objectives:

(1) Explaining something technically complex to readers who are not generally interested in "what's under the hood", but still nonetheless appreciate high-level "outreach" about what's actually possible today

(2) Defining the marginal benefits of limited research to a small group of peers who prefer to think only in terms of "what's under the hood", and thus are offended by higher-level connotations that they may associate with hollow marketing rather than rigorous science

Every scientist who communicates to the general public faces the above challenge in some form, and thus struggles to maintain both clarity and integrity all around... One of my inspirations was Christopher Olah (colah.github.io), who I think strikes a wonderful balance of being very careful about jargon, while being very literal.

In any case, a pernicious problem that we face here, and perhaps a premise for the OP's article, is the highly subjective, controversial, and polarizing definitions of artificial intelligence. Too often we forget that what we imagine to be artificial intelligence may feel trivial to another person, and vice versa.

I'd consider this a trick because initially you'd believe the script is what's doing the intelligent part. I think this is an example of impressive AI, but your friend's script isn't the AI, Google is. Imagine using Twilio to text the question to a real human and await their response. The script isn't the "intelligent" entity in either of those cases, just a broker.

Agreed. Look at speech recognition right up until a few years ago. Until Siri it just didn't work at all, and even Siri wasn't good enough to be useful when it first launched.

Now Google's voice search, and Amazon's Alexa are good enough to be actually useful. If you had asked an AI researcher 10 years ago if that would be the case they would probably have sniggered at you too.

Mmm...disagree.

Voice recognition has been okay for a long time, and while Siri/Cortana/OK Google are impressive displays for what they do, I don't really consider it AI. It's heavy keyword indexing combined with speech recognition, and all of the Voice Assistants on phones are still pretty limited in how they will interpret requests. That is, you can't really ask for a value judgement right now from the Assistants simply because they haven't really been written to do much else besides search for specific keywords and perform an action.

What I mean by this is suppose a scenario where you and friends are really hungry on a Friday and you're trying to think of what restaurant would have the shortest wait time. If you were to ask Siri, et. al. about it, unless the restaurant had a publicly accessible wait time that could be tapped for reference, the best the assistants could do is get you a list of open restaurants in yuor area. Your friends, however, probably can, with historical knowledge (times they've been there) and knowledge of current social trends (e.g., asian fusion is out, swedish fusion is in), could make a semi-educated guess as to which restaurant likely has the shortest wait. This is simply because we're recording this data subconsciously - the Assistants aren't because they've not been told to, nor do they even really have an understanding of it. If someone were to write up some parameters for the Assistants to consider it, they could start to do so (and honestly, probably should now that I think about it...).

But even 15ish years ago I was goofing around with voice control for my old boat anchor iMac and was pretty impressed as to how well it handled. It wasn't as fast as what Siri, et. al. do now, but the boat anchor didn't have an entire computing center to offload the processing workload to. There really isn't anything particularly special about what is happening right now with voice assistants except for the speed at which we get the results. It is very cool to be able to do it, but I would't call it AI. At the end of the day, you're still just having your vocalizations converted into a search query and getting the same limits to results you used to get in the past.

And to address the parent comment on Go "...requires an AI that displays intuition and use of heuristics", this is absolutely not in line with how the AlphaGo Researchers were describing it. Lee Sedol even started to get the hang of AlphaGo's prediction model and, if I remember correctly, it really didn't have any idea on what to do with a move that was a "wash". Lee's [1] victory came because of a move that was not evaluated correctly by AlphaGo - the move wasn't bad, in that AlphaGo couldn't capitalize on it, but it also wasn't "good" in that it didn't evaluate it as strong. AlphaGo's weakness was that it really didn't consider history at all when it came to moves, it only operated to rapidly assess the current condition of the board and try to determine the strongest path at the moment. It was a very impressive display to really hammer Lee Sedol like it did, but in interviews, Lee commented that he was starting to understand how AlphaGo made decisions, which was a problem in early games because he had no opponent to read, he had to understand an algorithm making decisions. This reddit thread has some okay breakdowns of how AlphaGo lost and how it decided moves: https://www.reddit.com/r/baduk/comments/4a7wl2/fascinating_i...

The entire point I'm trying to make is that while there are some very impressive Neural Network and Machine Learning tricks going on, there's also a lot going into making a really impressive curtain to hide all the computing behind. Right now, it's still the same decisions and techniques AI were doing years ago, just more efficient and faster. I wouldn't say that we have "strong AI", though I couldn't really define that if pressed. But I do know that right now Siri, AlphaGo, Watson, Cortana, OK Google and others are thinking only insomuch as a thinks. They're not really taking in and processing information as humans appear to.

[1] Just a note, I'm not sure on the naming formalities for Korean, so if I've madea faux pas with referring to Lee Sedol, it's from ignorance only.

Speech recognition has been ok for a long time if you happened to have an accent that worked fine with it.

I don't. I speak English with a relatively broad Scandinavian accent. Up until a few years ago, no voice recognition system I tested managed to recognise what I said with a high enough precision to be usable.

This "suddenly" changed over the last few years. And it may very well be that the approaches changed (e.g. more heavy emphasis on statistics and large scale feeding of voice data for bad recognition to people to help improve the models; offloading to data centres), but it did change for me at least. E.g. I can now not just have it recognise valid English words, but Youtube will recognise channel names that are mixes of words and letters/numbers, and Amazon Fire will recognise every actor name or movie title I trow at it with a precision rate I would not have expected just a 3-4 years ago.

It's to the point where the Google Docs speech recognition is almost usable for me too.

The fact that voice recognition now understands your accented English is not a sign of its increased ability to understand different accents: only of its increased coverage of specific accents which now include yours.

Not sure if I'm explaining that well enough. What I mean is, say Siri had a 90% accuracy in English before; that would be because it was trained with a lot of samples of English of many different accents. It wasn't trained with enough Scandinavian accents so it didn't understand them very well. Eventually more Scandinavian accented English examples became available and Siri's accuracy with Scandinavian accented English improved as a consequence.

But that doesn't mean that Siri is good at understanding English with any novel accent. So its improvement is not an improvement in terms of quality, only in terms of quantity: it can do more of what it did before, but nothing new.

Which is not to say that speech recognition doesn't work or that small improvements like that are useless, or anything like that, it's just to say there are still limitations and unless you're very careful it's easy to misunderstand what those are.

> Voice recognition has been okay for a long time

Ha what? That's just provably untrue. Just find an old Mac and try and ask it the time. A strong memory of mine is me and some friends discovering OSX's voice recognition system and trying to get it to work. It worked about 1/10th of the time, and the 'Tell me a joke' function was a particular challenge because it required you to get two recognitions to work in a row ("Tell me a joke", [knock knock], "Who's there?").

Remember those jokes techie people told about voice-activated photo copiers. That's how implausible it was that there was good voice recognition.

My experience is simply different. I suspect it's something to do with what the other person who replied to me said and that I happen to be from the Midwest of the United States with only a slight accent to give away my origin. The system was likely tuned around speech patterns like my own.

But I'm not going to deny what I had at the time. I could tell my Mac on OS 9 to open "Browser" and it could. I could tell it "Open control panel" and it could. I could ask it the time, or say "schedule", and it could tell me the time or open up my calendar for me.

This is why I'm somewhat dismissive of what we have today. In no way am I suggesting that voice recognition is the same now as it was 15 years ago (nor did I), but the method of operation and results are the same; the voice tools just have a larger vocabularly and thankfully can understand many other languages. But Siri and other Assistants are still pretty much just looking for keywords, just like the happy Mac was on OS 9. Siri sounds more natural than Victoria did, but if I say "Show my me daily schedule" to both Victoria and Siri, they're both going to hear "Schedule" and bring up my calendar.

I argue that on the go board, it did understand meaning and intent. It had a very nuanced appreciation for go positions, and demonstrated creativity. The critics were shocked, even felt like it had a personality.

But would it be as impressive on board of different size? The human would play on the same level, I imagine, would the AlphaGo?

19x19 is the normal go board size, I've never heard of anyone playing on a bigger board. Computer players for smaller boards (13x13 and 9x9) have been very good for a while, it's a much easier game on this smaller boards.

I think the parent's point was about whether the AlphaGo AI can generalize to a different board size (as a human player presumably can) rather than about what is the normal Go board size, or what board sizes are easier or harder.

The parent was suggesting that while a human might possess a general understanding of Go, performing comparably regardless of board size ("The human would play on the same level"), the AlphaGo AI in contrast might drop significantly in capability on a board of different size. In effect, the AlphaGo AI might just be a very well-fitted (perhaps even overfitted) machine-learned algorithm on a 19x19 board, lacking a fundamental "understanding" of the game that could generalize to a different board size.

Reply on news.ycombinator.com