IBM is not doing "cognitive computing" with Watson (2016) | Hacker News Reader

IBM is not doing "cognitive computing" with Watson (2016)

rogerschank.com

904 pointsdirtyaura435 comments

The author's particular gripe is that the Watson advertisements showing someone sitting down and talking to "Watson." They bother me as well (and did so when I was working at IBM in the Watson group) because they portray a capability that nothing in IBM can provide. Nobody can provide it (again to the author's point) because dialog systems (those which interact with a user through conversational speech) don't exist out side specific, tightly constrained, decision trees (like voice mail or customer support prompts).

If SpaceX were to advertise like that, they would have famous people sitting in their living room, on mars, and talking about what they liked about the Martian way of life. In that case I believe that most people would understand that SpaceX wasn't already hosting people on Mars.

Unfortunately many, many people think that talking to your computer in actually already possible, they just haven't experienced it yet. Not sure how we fix that.

It all goes back to how the Watson that played jeopardy - what the vast majority think of when they hear the word "Watson" - was a really cool research experiment and amazing advertising.

A lot of the people that pay for "Watson" probably think they're paying for something really similar to the Watson that beat Ken Jennings at Jeopardy! and cracked jokes on TV. They're paying for something that might use some of the same algorithms and software, but they're not actually getting something that seems as sentient and "clever" as what was on TV.

To me, the whole "Watson" division does seem like false advertising.

Yup. And they have sold it as a "potential solution" to everything data processing related. Hell, they even got NASA to pay for it:

https://www.space.com/35042-ibm-watson-computer-nasa-researc...

> Unfortunately many, many people think that talking to your computer in actually already possible, they just haven't experienced it yet.

Given how often another person can't correctly infer meaning when people talk, I doubt it will ever be how people imagine it.

Initially, I imagine it will be a lot of people trying to talk normally and a lot of AI responses asking how your poorly worded worded request should be carried out, choice A, or choice B. You'll be annoyed because why can't the system just do the thing that's right 90% of the time? The problem is that 10% of the time you're just not explaining yourself well at all and 10% is really quite a lot of the time, and probably ranges from a few requests to dozens of requests a day, and while being asked what you meant is annoying, having the wrong thing done can be catastrophic.

As humans, we get around this to some degree by modeling the person we are conversing with as thinking "what would they want to do right now", which is another class of problem for AIs I assume, so I think we may not get human level interaction until AI's can fairly closely model humans themselves.

I imagine we'll probably develop some AI pidgin language that provides more formality that as long as we follow it AI voice interactions become less annoying (but you have to spend a bit of time before to formulate it).

Then again, maybe I'm way off base. You sound like you would know much better than me. :)

In a previous job I looked into the viability of pen computing, specifically which companies had succeeded commercially with a pen-based interface.

A lot of folks are too young to know this, but there was a time when it was accepted wisdom that much of personal computing would eventually converge to a pen-based interface, with the primary enabling technology being handwriting recognition.

What I found is that computer handwriting recognition can never ever be good enough to satisfy people's expectations. Why? Because no one is good enough to satisfy most people's expectations for handwriting recognition, including other humans, and often even including themselves!

Frustration with reading handwriting is a given in human interaction. It will be a given in computer interaction too--only, people don't feel like they need to be polite to a computer. So they freely get mad about it.

The companies that had succeeded with pen computing were those that had figured out a way to work around interpreting natural handwriting.

One category was companies who are specialized in pen computing as an art interface--Wacom being the primary example at the time. No attempt at handwriting recognition at all.

The other was companies who had tricked people into learning a new kind of handwriting. The primary example is the Palm Pilot's "Graffiti" system of characters. The neat thing about Graffiti is it reversed customer expectations. Because it was a new way of writing, when recognition failed, customers often blamed themselves for not doing it right!

We all know what happened instead of pens: touchscreen keyboards. The pen interface continues to be a niche UI, focused more on art than text.

It's interesting to see the parallels with spoken word interfaces. I don't know what the answer is. I doubt many people would have predicted in 2001 that the answer to pen computing was a touchscreen keyboard without tactile keys. Heck people laughed about it in 2007 when the iPhone came out.

But I won't be surprised if the eventual product that "solves" talking to a computer, wins because it hacks its way around customer expectations in a novel way--rather than delivering a "perfect" voice interface.

> The other was companies who had tricked people into learning a new kind of handwriting [...] It's interesting parallels with spoken word interfaces. I don't know what the answer is.

Well, here's a possibility: we'll meet in the middle. Companies will train humans into learning a new kind of speech which is more easily machine-recognisable. These dialects will be very similar to English but more constrained in syntax and vocabulary; they will verge on being spoken programming languages. They'll bear a resemblence to, and be constructed in similar ways to, jargon and phrasing used in aviation, optimised for clarity and unambiguity over low-quality connections. Throw in optional macros and optimisations which increase expressive power but make it sound like gibberish to the uninitiated. And then kids who grow up with these dialects will be able to voice-activate their devices with an unreal degree of fluency, almost like musical instruments.

I'd just like to see a voice interface that has any attempt at natural error correction.

We do this in a hundred ways in English, emphasis, meta-words, too many to fold into a system. But with a little user prodding, I think you could cover enough of this ground to still sound almost natural.

  Asst.: Would you like me to read that back to you?
  User: Yes
  A: The click brown ox...
  U: The _quick_ brown _fox_...
  A: Ok. The the quick brown fox jumped over the lazy log. 
  Is that correct?
  U: No.
  A: Which part [is incorrect]?
  U: The the quick
  A: The the quick. What should it be?
  U: The quick.
  A: The quick brown fox jumped over the lazy log. Is that 
  correct?
  U: No. Lazy dog.
  A: Lazy log?
  U: No. Lazy dog.
  A: Spell "log." <Or "Spell" + recording of user's voice saying ambiguous word>
  U: D-O-G.
  A: Ah. The quick brown fox jumped over the lazy dog. Do 
  I have it now?
  U: Yes.
  A: Say 'send' to send message.

Agreed that voice systems are an incredibly hard problem, but I suspect rule-based natural language correction systems would go a long way.

I'm not so sure. Maybe I'm naive but I could see imitation and reinforcement learning actually becoming the little bit of magic that is missing to bring the area forward.

From what we know, those techiques play an important roles in how humans themselves go from baby glibberish to actual language fluency - and from what I understand there is already a lot of research ongoing how reinforcement learning algorithms can be used to learn walking cycles and similar things.

So I'd say, if you find a way to read the non-/semiverbal cues humans use to communicate understanding or misunderstanding among themselves and use those as penalties/rewards for computers, you might be on a way to learn some kind of languages that humans and computers can communicate with.

The same goes into the other direction. There is a sector of software where human users not only frequently master highly intricate interfaces, they even do it out of their own motivation and pride themselves on archieving fluency: Computer games. The secret here is a blazing fast feedback loop - which gives back penalty and reward cues in a similar way to reinforcement learning - and a carefully tuned learning curve which starts with simple, easy to learn concepts and gradually becomes more complex.

I would argue, if you combine those two techniques - using penalties and rewards to train the computers and communicate back to the user how well the computer understood you, you might be able to move to a middle-ground representation without it seeming like much effort on the human side.

The first generation Newton was insanely good at recognizing really terrible cursive handwriting and terrible at recognizing tidy or printed handwriting. The second generation Newton was very good at recognizing tidy handwriting (and you could switch back to the old system if you were one of the freaks whose handwriting the old system grokked). The Newton also had really nice correction system, e.g. gestures for changing the case of a single character or word. In my opinion the later Newton generations recognized handwriting as well as you could expect, when it failed you forgave it and could easily correct the errors, and was decently fast on the 192MHz ARM CPUs.

Speech is way, way harder.

I got a two-in-one Windows notebook and I was really, really impressed with the handwriting recognition. It can even read my cursive with like 95% accuracy, which I never imagined. That said, it's still slower than typing.

I've always thought it'd be nice to be able to code at the speed of thought - you can often envisage the code or classes that'll make up part of a solution, it's just so slow filtering that through your fingers and the keyboard; if the neural interface 'thing' could be solved that'd be a winner I think, both for coding and manipulating the 'UI'.

It seems within the bounds of possibility to me (from having no experience in the field ;), if patterns could be predictably matched in real-time or better time than it takes to whack the keyboard or move the mouse (can EEG or FMRI make predictable sort of patterns like that?).

This comment is way too long to justify whatever point you are making. First of all, how long ago do you mean? Are you 100 years old and people in the 1940's thought computers would be operated by pens?

But also, when my college had a demo microsoft tablet in their store in 2003, I could sign my name as I usually do (typical poor male handwriting) and it recognized every letter. That seems pretty good to me.

I also had a Palm Pilot then and Grafitti sucked, but that's just one implementation.

I think you have a good point. Something that was fascinating for me was that one of the things the search engine did after being acquired by IBM was looking at search queries for common questions people would ask a search engine. Taking 5 years of search engine query strings (no PII[1]) and turning them into a 'question' that someone might ask.

What was interesting is that it was clear that search engines have trained a lot of people to speak 'search engine' rather than stick with English. Instead of "When is Pizza hut open?" you would see a query "Pizza Hut hours". That works for a search engine because you have presented it with a series of non stopword terms in priority order.

It reminded me of the constraint based naming papers that came out of MIT in the late 80's / early 90's. It also suggested to me ways to take dictated speech and reformat it into a search engine query. There is probably a couple of good academic papers in that data somewhere.

That experience suggests to me that what you are proposing (a pidgin type language for AI) is a very likely possibility.

One of the bits that Jimmy Fallon does on the Tonight Show is an interviewer who asks people a question but uses nonsense words that sound "right" in the question. The most recent example was changing the question

"Do you think you will see Avengers: Infinity War"

With things like "Do you think you will see Avengers something I swear." Similar sounds and people have already jumped to the question with enough context that they always answer as if he had said "Infinity War." How will conversational systems deal with that, I have no idea.

> One of the bits that Jimmy Fallon does on the Tonight Show is an interviewer who asks people a question but uses nonsense words that sound "right" in the question. The most recent example was changing the question

Yeah, that's exactly what I'm talking about. I suspect this works because the interviewees have a model of what they think Jimmy Fallon is going to ask, and when hearing a question, if they can't make out words or they sound wrong, the simplest thing is to assume you heard them wrong and think what it's likely he would have asked. To that, the answer is fairly obvious.

We've all experienced those moments when we're caught off guard when this fails. For example, you're at work and someone you're working with says something that sounds like it was wildly inappropriate, or out of character, or just doesn't follow the conversation at all, and you stop and say "excuse me? What was that?" because you think it's more likely you heard them wrong than to believe you fuzzy matched their statement "Hairy Ass?" and you were correct.

I've been using search engines since Lycos and before, so I reflexively use what you call "search engine" language. However, due to the evolution of Google, I've started experimenting with more english-like queries, because Google is getting increasingly reluctant to search for what I tell it in a mechanistic way, often just throwing away words, and from what I've read it's increasingly trying to derive information from what used to be stopwords.

One thing that is very frustrating now is if there are two or three things out there related to a query, and Google prefers one of them, it's more difficult than ever to make small tweaks to the query to get the one you want. Of course, sometimes I don't know exactly what I want, but I do know I don't want the topic that Google is finding. The more "intelligent" it becomes, the harder it is to find things that diverge from its prejudices.

There was an article in The Atlantic recently that remarked upon this.

The Atlantic: Alexa Is a Revelation for the Blind. https://www.theatlantic.com/magazine/archive/2018/05/what-al...

Then I stop myself. Isn’t it possible that he expects Alexa to recognize a prompt that’s close enough? A person certainly would. Perhaps Dad isn’t being obstreperous. Maybe he doesn’t know how to interact with a machine pretending to be human—especially after he missed the evolution of personal computing because of his disability.

...

Another problem: While voice-activated devices do understand natural language pretty well, the way most of us speak has been shaped by the syntax of digital searches. Dad’s speech hasn’t. He talks in an old-fashioned manner—one now dotted with the staccato march of time. “Alexa, tell us the origin and, uh, well, the significance, I suppose, of Christmas,” for example.

>One of the bits that Jimmy Fallon does on the Tonight Show is an interviewer who asks people a question but uses nonsense words that sound "right" in the question.

There was a similar thing going around the Japanese Web a few years ago, where you'd try to sneak in soundalikes into conversation without anybody noticing, but the example I remember vividly was replacing "irasshaimase!" ("welcome!") with "ikakusaimase!" ("it stinks of squid!").

Humans are really good at handling 'fuzzy' input. Something that is supposed to have meaning, and should if everything were heard perfectly, but which has some of the context garbled or wrong.

At least, assuming they have the proper context. Humans mostly fill in the blanks based on what they /expect/ to hear.

Maybe someday we'll get to a point where a device we classify as a 'computer' hosts something we would classify as an 'intelligent life form'; irrespective of origin. Something that isn't an analog organic processor inside of a 'bag of mostly salty water'. At that point persistent awareness and context(s) might provide that life form with a human-like meaning to exchanges with humans.

No, you're completely on the right track.

People usually judge AI systems based on superhuman performance criteria with almost no human baseline.

For example, both Google Translate and Facebook's translation system could reasonably be considered superhuman in performance because the singular systems can translate into dozens of languages immediately more accurately and better than any single human could. Unfortunately people compare these to a collection of the best translators in the world.

So you're exactly on track, that humans are heavily prejudiced against even simple mistakes that computers make, yet let consistent continuous mistakes slide for humans.

>Unfortunately people compare these to a collection of the best translators in the world.

I don't think that's really true. They're comparing them to the performance of a person who is a native speaker of both of the languages in question. That seems like a pretty reasonable comparison point, since it's basically the ceiling for performance on a translation task (leaving aside literary aspects of translation). If you know what the sentence in the source language means, and if you know the target language, then you can successfully translate. Translation isn't some kind of super difficult task that only specially trained translators can do well. Any human who speaks two languages fluently (which is not rare, looking at the world as a whole) can translate between them well.

This is well said, but one reason this double-standard is rational is that current AI systems are far worse at recovery-from-error than humans are. A great example of this is Alexa: if a workflow requires more than one statement to complete, and Alexa fails to understand what you say, you are at the mercy of brittle conversation logic (not AI) to recover. More often than not you have to start over or worse yet execute an incorrect action, then do something new to cancel. In contrasts, even humans who can barely understand each other can build understanding gradually because of the context and knowledge they share.

Our best AIs are superhuman only at tightly scoped tasks, and our prejudice encodes the utility we get from the flexibility and resilience of general intelligence.

I might use this in future critical discussions of AI. “It’s not really intelligent.” Yeah, well, neither am I. On a more serious note, it seems obvious to me that technology is incremental, and we are where we are. Given 20 more years of peacetime, we’ll be further along. When VGA 320x200x256 arrived it was dubbed photorealistic. I wonder what HN would have had to say about that.

Being able to do many things at a level below what trained humans can isn't what any reasonable person would call superhuman performance. If machine translation could perform at the level of human translators at even one pair of languages (like English-Mandarin), that would be impressive. That would be the standard people apply. But they very cleary can't.

> I imagine we'll probably develop some AI pidgin language that provides more formality that as long as we follow it AI voice interactions become less annoying (but you have to spend a bit of time before to formulate it).

There's been some tentative discussion of using Lojban as a basis for just that!

https://en.wikipedia.org/wiki/Lojban#Lojban_as_a_potential_m...

http://www.goertzel.org/new_research/Loglish.htm

>> I imagine we'll probably develop some AI pidgin language that provides more formality that as long as we follow it AI voice interactions become less annoying (but you have to spend a bit of time before to formulate it).

That's what Roger Schank is advocating against in his article: artificial intelligence that makes humans think like computers, rather than computers that think like humans.

Personally, I think it's a terrible idea and the path to becoming like the Borg: a species of people who "enhanced" themselves until they lost everything that made them people.

Also: people may get stuff wrong a lot of the time (way more than 10%). But, you can always make a human understand what you mean with enough effort. Not so with a "computer" (i.e. an NLP system). Currently, if a computer can't process an utterance, trying to explain it will confuse it even further.

What's bad about it is that it leads to a "market for lemons" for anyone working in the A.I. field in either engineering or sales. People see so much bullshit that they come to the conclusion that it is all bullshit.

I think this is good.

In a world that moves too quick to have review/ratings companies for everything, you should come in skeptical and be proven wrong.

Its up to the merchant to make a great product, its up to you to compare and make good decisions.

Sure, but Watson isn’t really a product and IBM isn’t really a merchant here. There’s never going to be a Watson review anywhere because (AFAICT) Watson is just a marketing name for a wide range of technologies, none of which do any of the stuff shown in the TV commercials. That’s the problem.

I don’t even know who those ads are targeting. Anyone who knows anything at all about this stuff will be dismayed at the sheer BS of it all. Everyone else seeing the ads isn’t in a position to steer customers towards IBM for all their AI needs. They really need to kill the whole ad campaign, surely it’s doing IBM more harm than good.

Remember these ads?

https://youtu.be/x7ozaFbqg00

No, it's bad because it drives down the potential market value for good products that could become available in the market because the market expects that all AI is inferior to expectations.

How do you "compare" multimillion dollar custom IT installations, before you buy?

The "market for lemons" theory proved that used car dealerships can't exist. And yet they do. People understand that ads are just a hint and a tease, not a precise value proposition.

That isn't the conclusion of the market for lemons. It predicted that the price of used cars would be substantially depressed due to unknown quality, especially when the seller had little reputation. Considering the huge depreciation a new car experiences as soon as it is driven off the lot and that prices are lower when buying from individual sellers rather than used car dealerships, there is plenty of evidence that the theory was correct.

What I find interesting is that free Carfax reports are becoming pretty common, which seems like a mechanism to deal with the problem of a market with only lemons. But I was browsing cars online and started looking for "good" ones, defined as no accidents and regular oil changes at the dealer, and interestingly they were almost non-existent. So in fact, the proposition that cars for sale will be lemons seems to be true even though the compensatory mechanisms exist. It would appear maybe buyers don't utilize information to their advantage, which reminds me of a recent article about how investors don't seem to digest all the information in SEC filings, even though orthodox market theory assumes prices incorporate all public data.

This is not just Watson and IBM. Many, many people in AI make grandiose claims and throw around big words like "Natural Language Understanding" "scene understanding" or "object recognition" etc.

And it is a very old problem, at least from the time of Drew McDermot and "Artificial Intelligence meets Natural Stupidity":

https://homepage.univie.ac.at/nicole.rossmanith/concepts/pap...

From which I quote:

However, in AI, our programs to a great degree are problems rather than solutions. If a researcher tries to write an "understanding" program, it isn't because he has thought of a better way of implementing this well-understood task, but because he thinks he can come closer to writing the _first_ implementation. If he calls the main loop of his program UNDERSTAND, he is (until proven innocent) merely begging the question. He may mislead a lot of people, most prominently himself, and enrage a lot of others.

This is Marketing 101 though. It's easier to sell someone a dream or some emotional state than it is to sell an actual product/thing that you have. I'd give people a little more credit. Adults know what advertisements are and that they're all phony.

I don't disagree, although I'd be a bit more nuanced than that. It is like herbal supplements, you can say how great they make you feel but you have to avoid making medical claims that you can't back up.

I don't think anyone would have flinched at setting the stage like "At IBM we're working on technology so that some day your interaction with a computer can be as simple as sitting down and having a conversation."

Setting expectations you cannot meet is something you try to avoid in Marketing. Because you get angry op eds like the one that kicked off this conversation.

>I don't think anyone would have flinched at setting the stage like "At IBM we're working on technology so that some day your interaction with a computer can be as simple as sitting down and having a conversation."

Like those old AT&T commercials (available on YouTube) where AT&T made all kinds of predictions about the future that seemed crazy in the 1980's, but are reality 35 years later. Video calls, instant electronic toll booth payment, etc...

/ R.I.P, AT&T.

IE, unlimited data plans where unlimited is defined as de-prioritized after X GB per billing month.

I hate tmobile for this reason.

Totally crap, especially since tmobile runs ads that have pink background and 1 word on the screen:

UNLIMITED

No. Its not unlimited. I know because I paid 25 extra dollars for a month for their international plan. Truly unlimited.

A lot of non-technical decision makers don't think this is phony and are making life very difficult for engineers in their companies we will all remember this though. For the future.

There was a time when I'd introduce people to 'Eliza'-type programs. Those programs stashed phrases typed in by users. When new users typed in a phrase, the programs would parrot back the stashed phrases ... based on really crude algo's, or even random selection.

Nonetheless, I watched people get really worked up about what the computer was 'saying'. Partly because of the sarcastic stuff people would say, partly because of their expectations about 'cybernetic brains'.

Now the 'Cyc'le is back. And people actually working on this really hard problem are not helped by dumb marketing.

> Not sure how we fix that

How about prosecuting them for false advertising? Seems like a classic case, albeit in a new domain.

We could also classify fake ads using machine learning...

Red Bull doesn't actually give you wings. Ads are metaphors.

No, the Red Bull ad _uses_ a metaphor.

False advertising is claiming to be something you are not, raising false expectations.

One way to fix it is not to make outright false advertisements.

Or at least require cigarette style warning labels:

Warning, this advertisement is a lie. Believing this advertisement may harm your cognitive health.

Warning, this statement is false.

> I believe that most people would understand that SpaceX wasn't already hosting people on Mars.

You overestimate people's understanding of space. Many people thought "The Martian" was a true story like "Apollo 13".

How many is “many” though? I’m assuming slightly more than thought Star Wars was real, and slightly fewer than think Iron Man was real. That’s not ignorance, that’s mental illness.

> Not sure how we fix that.

Easy: just say that all Watson systems have the hardware needed for cognitive computing and full dialog support.

It’s a pretty serious problem if public company executives who choose how to spend investors dollars actually believe SpaceX’s famous people are on Mars. Then they pay SpaceX to start carrying goods to Mars to sell to the Martians. SpaceX obliges with multi-million dollar contracts, knowing they don’t have a way to get to Mars but have some rockets that can put satellites into orbit (not the same).

We’d all see the problem with such a situation. People with a 6th grade knowledge on space travel understand that the commercials were BS marketing and we should fire the executives for not knowing the same or doing their due diligence. We’d be mad at SpaceX for taking advantage of companies and failing to reach Mars as promised.

In my limited understanding, isn't this just the state of A.I. in general? I don't know of anyone trying to solve the general intelligence problem. Everyone's just finding formula for specific applications and using M.L. to curve fit.

Creating an impression of a capability without legally promising it is the essence of marketing, especially in the corporate IT consulting world which is 90% garbage and hugely expense.

To be fair, you can kinda talk to Google assistant in a "question response" type capacity and it does pretty well.

You can also kinda talk to ELIZA.

Google Assistant, Apple Siri and Amazon Echo are all quite fancy natural language parsers. After parsing, it's just a group of "dumb" (wo)man-made widgets.

There not entirely true. Google Search will answer a question with an automatically extracted excerpt from a website page. Watson did the similar with its Jeopardy system

Is that not parsing and keyword searching? Once again there's no "cognitive computing" going on.

Which is a really weird and nebulous thing to define, by the way. I think we aren't going to have super convincing cognitive computing until we have something approaching AGI. Which of course is waaay different from the AI and machine learning that is popular today, despite most laypeople conflating any mention of AI with AGI. Of course, when IBM is making an ad, they are largely aiming at laypeople.

Only in some cases, supposedly as a fallback.

Search and command interfaces are doable (within reasonable expectations) dialogs just aren't without eye popping engineering challenges (like 100k branches hand engineered onto trees + lots of deep learning.)

Also we (science) know shockingly little about real dialogs and their challenges as far as I can find out - but I am not a linguist and am angling here fore some killer references !

False advertising laws?

Kinda how Tesla advertises autopilot as a self driving car that's safer than human drivers?

> Full Self-Driving Hardware on All Cars

> All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver.

https://www.tesla.com/autopilot/

Am I extreme in feeling someone should go to jail for that? It was bad enough when they were originally advertising it, but now that they're defending it even after people died...ugh.

https://www.theverge.com/2018/3/28/17172178/tesla-model-x-cr...

Are you for real? Statistics prove that Tesla is right on this. I’m not saying human life has no value, and I do empathise with the people who lost someone due to a Tesla crash, but we can see what is the percentage. It IS lower compared to human error, and yes that includes DUI and every other time it was a human error.

Tesla is safer for the community based on the statistical data. Also it is safer than a human driving. I’m not saying it will never crash just that it will crash less than a human.

Maybe another way to compare it would be to an autopilot in airplanes. Sometimes it fails so we might as well be angry at the airline companies if they advertise it. But a pilot would make more mistakes than the autopilot with the current workload.

Side note: if you are going to downvote you can comment on why. Negative feedback is very useful and helps build better argumentative skills.

Claiming that the hardware is ready and we're just waiting on the software is at best a vacuous claim and worse, a fraudulent one. I can hammer two web-cams onto a rotating plank and claim that's "full self-driving capability". Just look at the eyes in my head! Now I just need to iron out some kinks in the software...

That is a much better example. With the twist that believing it can get you killed.

"Self-driving" is not the same as "Never going to crash."

People just assume that if a computer does something, it's going to be perfect. It isn't.

Most humans can't even "self-walk" without occasionally stumbling and falling. Does that make human locomotion dangerous and false?

It's not just them. A lot of people think that because NNs can do object identification that self driving cars are almost here. It's similar to thinking that word recognition and pattern matching are going to give us conversational AI. Things like that are prerequisites for AI, but they are hardly the whole thing. Companies are spending billions on this stuff and I hope they're just hedging in case it pans out rather than planning their future around it.

Unfortunately people take that marketing literally https://www.independent.co.uk/news/uk/crime/tesla-autopilot-...

"...have the hardware needed for...". Nothing about the full capability. So no actual claim about the capability, just that the hardware is in place.

Still a bit deceptive, to me. Reminds me of "...electroplated in 16 carat gold...".

But it’s technically correct: the hardware is present. The software isn’t there yet, that’s true. So is this a “lie by omission”? Reading the whole page, there’s this quote: “It is not possible to know exactly when each element of the functionality described above will be available.” What else needs to be said?

At some point, critical thinking is required to navigate the world. The customer should ask “and when will the software catch up?” And I’ll concede that the overwhelming majority of the population doesn’t think like that. Should Tesla update the site with statements to confirm the negative cases? e.g. “Our software can’t yet do X, Y, Z, etc”

Hardware. That’s the keyword, and it might very well be true. Tesla doesn’t claim their software is capable of full self-driving yet and in fact the system will scream at you and stop the car if you take your hands off the steering wheel for too long.

We fix that by building conversational AI ;)

The most ingenious trick that the IBM marketing department pulled was to get non-technical (and probably even technical people, judging by this thread) to think that Watson is some kind of singular thing. Like that it’s a single big neural network with different APIs on it, or something. I honestly think that’s what most people think Watson refers to.

Watson is like Google Cloud Platform. It’s just a name for a platform with a bunch of technologies.

E.g. Watson Natural Language Understanding was previously AlchemyLanguage. It was just rebranded.

It’s very clever though, I’ll give them that. Use a human name so it has all the anthropomorphic connotations and let people think it’s some kind of AI learning things.

I'm not even convinced Watson is a platform. My impression is that it's just a consulting division of the company that deploys teams to build solutions that are in some way related to AI, with each solution or implementation potentially being completely unique from the ground up. Perhaps someone from IBM can correct me though.

I'm currently sitting in a meeting about implementing the Watson Enterprise Search product in my company and that is more or less the impression I've gotten. They sell it as a platform that is easy to customize and then once you're in they bill you tons of hours to help you because the system is indecipherable and poorly documented.

And that, folks, is enterprise software services/consulting in a nutshell!!

They sell it as a platform that is easy to customize and then once you're in they bill you tons of hours to help you because the system is indecipherable and poorly documented.

So pretty much like any major enterprise system from the likes of IBM, SAP, Oracle, ...

James we can all see you with your phone under the table

Sounds like every IBM product: WebSphere, Tivoli, WSSR, RAD, fuck even AIX. Many of those can be replaced with open source tools at a fraction of the cost and at a huge increase in performance.

Customer collaboration at its finest. Documentation checks out too.

How did the meeting go?

Watson is a brand name. Specifically It's the Machine Learning brand name. Watson Developer Cloud is the product suite and it's just a set of pre-trained classical, machine learning, and deep learning based APIs for a variety of tasks. NLU(UIMA) text identification, NLC(Fuzzy String Matching), Visual Recognition, Tone Analysis(VADER), Discovery (Document Database + NLU + Knowledge Graph), Speech (STT/TTS), Text Translation (Literal not Semantic), Assistant (Conversational State Engine with embedded linguistic neural net). We're ahead in some aspects and a bit behind in others. There is also a generic Machine Learning Service which allows you to train Classical or Deep Learning algorithms and push them to a rest endpoint for production use. Ultimately the "Watson" from jeopardy was sliced up and pieces stuffed into various products. Anything with a smattering of AI/ML gets the Watson brand on it. I personally hate the Watson commercials as people who don't know anything about the subject think Watson is this singular sentient entity. Those who do know about AI/ML know we have the same general tech as everyone else. One benefit we do have though is petabytes of training data and expertise in just about every line of business on the planet.

I only know specifically about the NLP stuff, e.g. Natural Language Understanding (AlchemyLanguage), Natural Language Classification (it's just a multi-label text classifier) and Watson Knowledge Studio (Basically allows you to create your own named entity recognition classifier (NERC), also supports relations and co-reference resolution. You manually hand-annotate examples through a Web UI).

So by platform I mean, lets say you train a NERC model using Watson Knowledge Studio. Obviously this model has to be "deployed" somewhere so you can call it using an API. They host it for you and they bill you per API call. Anyone can go create their own entity type system and manually annotate a training dataset. So it's definitely a re-usable platform, you don't need to pay for any IBM consultancy to use it. I found that the NLP offerings have many problems, and that the documentation alone is not enough to help resolve all of them. So eventually, IBM will just tell your employer you're stupid and that's why it's not working as it should and you should pay IBM to come in.

But make no mistake, these are all just standard machine learning tools that have been "packaged" so end-users can use them through a web front end. It is in no way, whatsoever, getting any input from any AI/Neural Network/Database/whatever you want to call it/ thing called "Watson".

I personally think it's disingenuous because when people hear Watson they think Jeopardy and they think that somehow that technology is involved when they use any of the Watson.* products.

> I personally think it's disingenuous because when people hear Watson they think Jeopardy and they think that somehow that technology is involved when they use any of the Watson.* products.

The use of the Watson name is a deliberate attempt to take advantage of the Jeopardy game. It's a name that has cachet, and I've seen just enough of the marketing perspective to know that marketing will push very hard to reuse a successful name.

I used to work for IBM, on a backend service used by various Watson (and non Watson) branded projects.

I think federation would be a better term. There was a core set of APIs and hardware that might be called "Watson proper" but each market segment would be handled by a different organization. And then there was the proliferation of odd ball things out of research or little groups looking for growth/stability that get Watson branded.

Sometimes we'd be the first time a team relaizes there is already something doing what they've been building.

At my previous firm I worked with a pre-sales engineer who was formerly at IBM Watson before working at the firm. This is essentially it. Implementations of Watson were no different than doing an ERP project.

I knew people who had their division pay for Watson as a way to get their business AI and data science needs fulfilled without hiring developers outside their price range.

Eventually they scrapped the project because it not only took a ton of employee time to talk with IBM's team to get it set up and working, it also cost a significant chunk of money and wasn't as good as what the people who already worked at the company thought they could do themselves.

With all due respect to the people that work at IBM, I just can't imagine IBM's sales and consulting cultures to work well with deploying AI. I don't know firsthand, but from what I've heard and what I would guess anyway, a lot of the people selling Watson and actually on the front lines working with it probably aren't that knowledgeable about AI/ML/whatever. I just don't see how you could determine a project's feasibility or effectiveness without having a sharp conceptual knowledge of the actual AI algorithms and what potentially what kind of data is needed to make them shine.

Suppose a university admissions department offers paper surveys to prospective students at the end of on-campus tours. In an effort to improve admitted student yield (the percentage of students that actually attend the university after being accepted), the university wants to be able to scan these surveys' text digitally and then perform sentiment analysis to determine how excited the student is about attending the university, or more directly, how likely the student is to matriculate. The university doesn't have any people capable of doing this, so they get into contact with Watson.

How much will the salespeople at Watson pry into the questions of the survey, demographics and culture of the school, or the sample size? Will they ask about statistics such as acceptance rate, yield, and which students are most likely to matriculate (based on quantifiable metrics)? Even the type or color of paper and text field sizes on the surveys on could affect the feasibility of the project regarding OCR, or bias the responses toward short answers. I would argue that a lot of knowledge about the project would be necessary before a sales quote or even the feasibility of the project itself could be considered, but would a salesperson know to ask these question? Would they even be incentivized to ask? Would the consultants know that certain questions could make OCR hard or sentiment analysis a wash? Would a statistician be consulted to see if the same or better results could be obtained from simple analysis of GPA, ZIP, and test scores?

I'm sure everybody at Watson is pretty technically competent - and to be sure, I'm sure for most consulting and sales that IBM does, I wouldn't have to make the following qualification. But to be brutally honest, I think the type of people who are familiar enough with AI to be the person you want working at Watson in consulting and sales probably are using those skills as developers and data scientists. And even then, again with all due respect to IBM employees (and I know IBM puts out a lot of great research), those people might not also be at IBM either.

some history on the topic:

at the beginning there was the one true watson. watson was a way to process, correlate and provide indsight on a corpus of knowledge expressed in natural language. the technology was good but had one major weakness: the knowledge extraction part had a large bulk of manual labor needed to weed out the noise from the relevant part, because to a processing engine each bit of information is equal to every other bit of information. so you needed domain expert to proviede an initial tuning and after that watson was a good solution for the problem statement.

this process however required non technical domain expert to be working closely with the watson analysts at the tuning for an unspecified and quite long amount of time, comopounding the already astronomical costs of the solution itself

now as you might imagine like any other company ibm has a lot divisions - cloud, services, intelligence etc. the watson division due the large research costs and the few clients that were able to afford and make use of the tech was scoring too much quarters in red.

ibm is also a financial company, so they did what they usually do when one division needs padding: they started moving everything remotedly related to intelligence and analytics under the watson moniker, to drive up quarterly reports. this had the side effect that the watson marketing is a clusterfuck of overlapping and unrelated solutions that more pften than not don't even work togheter natively, but are presented as a whole ecosistem.

now, of course anyone trying to make sense of the whole thing is going to be faced with all kind of claims against all kind of slutions, without any idea of what does what.

but originally the only omission from marketing was what watson actually required and how and what it could give back to a company. but the problem is... it's the kind of solution that you have to build to see where it goes. you cannot be sure of the results from the beginning.

Same thing Salesforce is doing with Einstein right now. Means that internally, when someone says a customer wants to talk about Einstein, people are all left wondering which one.

This is not ingenious in my opinion ... I’ve been at a company for a year that had bought into “Watson” — when I suggested alternatives to the specific apis being used was told in no uncertain terms that they had been consulted and they were going with “Watson” ...

Now a year later I’m finally being asked to clarify “what is Watson” so that the decision makers can better understand the techniques being used rather than the fantasies about what was being used that they were encouraged to develop through misleading marketing and consultants ...

I remember when Microsoft .NET came out and I hated it because they named it something that had no relation to what it was. The product had nothing to do with the Internet, but the Internet was a big new fad back then and marketing wanted to latch onto that.

Today .NET is a great product ecosystem and a huge success, except for the horribly awkward name.

I don't know if they'll ever regret naming it Watson, but latching onto the AI craze isn't necessarily a losing strategy as long as the products are good and successful. Even if they have nothing to do with AI.

It seems to be a common theme with projects in big companies: somebody comes up with a good project idea and a catchy name. As it gets resources and management attention, other departments re-brand their long-time toy projects with being a substantial part of the 'catchy-name' project's vision. At some point nobody knows anymore what it was all about. 'SDN', 'Cloud', 'Watson', '.NET' are all examples of this.

I attended the IBM Connections conference in Vegas shortly after the Jeopardy! thing and just after IBM started using Watson as a brand under which it lumped a bunch of analytics products. From questions and comments made, during some of the sessions I attended, it became clear that large portion of the attendees (mostly the business people) wrongly assumed that the technology that won Jeopardy! was now being used inside everything labeled with "Watson". People were very excited by this. I never heard anyone from IBM making any attempt to try and rectify this misconception, they just smiled, nodded and played along. I disliked IBM and their corporate marketing BS even more after this.

Totally. Especially because it was on Jeopardy, further reinforcing the idea that it is a single box. Maybe it was then, but that's definitely an impression that's stuck with me since.

I was convinced it was singular when they had a big mainframe sitting there to play He. Today I strongly believe it's simply a brand.

But I bet you that when the singularity arrives, it will be named "Watson".

I briefly worked with a Watson team on a cool idea to map a person's 'knowledge space' (or probable knowledge space given their background) against Watson's knowledge space and guide them to relevant learning materials and journal articles and the like.

The idea was to save people time so they aren't rehashing stuff they know down pat or jumping ahead into material they cannot understand but, instead, find that next step into what they almost know. The idea from there would be to let them specify where they want to go and guide them, step by step, exposure by exposure, to that summit.

In a few days, it turned into Just Another News Article Recommendation Engine based on interest and similar profiles with other clients. Yawn.

In a similar vein, I find a machine beating a grand master fairly boring. Show me a machine that can teach someone to become a master (or even a very good amateur) and we can talk.

That is, don’t beat the game, beat the opponent, understand why, and then model an adaptation strategy for the opponent (teach).

> In a similar vein, I find a machine beating a grand master fairly boring.

I know a lot of people felt that way even before it happened. It feels like an accounting problem rather than intelligence. But, it's fun to remember when it was thought by some very smart people that chess required more accounting than was ever possible by a machine, so beating a grand master would have to be a demonstration of intelligence.

> don’t beat the game, beat the opponent, understand why, and then model an adaptation strategy for the opponent (teach).

Yeah that would be closer. My favorite Turing test, if you will, is whether the AI can tell you you're asking it the wrong question. If Watson got bored of beating grand masters at chess and started refusing to play, maybe then a case could be made it was reasoning.

A lot of AI "milestones" were actually caused by improvements in computing itself, IMO. That is, they weren't facilitated by novel algorithms or engineering solutions specific to the milestone itself. They kind of just happened, either because computers got faster (e.g. disk I/O, ram size, processor speeds and caching) or because someone finally decided to throw enough distributed computation at the problem.

People knew about the main algorithms that Deep Blue used (alpha-beta minimax with heuristic evaluation) for decades before Deep Blue existed. It was simply a matter of waiting until IBM found marketing value in dedicating a top 300 world super computer to beating Gary Kasparov

> Watson got bored of beating grand masters at chess and started refusing to play

That's when you reboot it, if that doesn't fix it good old persuasive maintenance techniques can also come into play

"I'm afraid I can't let you do that, Dave" - HAL

I'm interested in the original idea. Can you expand on how your ideal system would function?

Well, the hope was that Watson, having explored and built a connected knowledge graph from various sources, could ask probing, adaptive questions to find out where a person landed.

So, say I'm an undergrad at a good university and I tell the system "I'm interested in Computer Science. I am particularly interested in Scientific Computing and would like to get to a graduate level of knowledge."

The system might ask "Sort the following operations by their worst-case run-time...". Then, if they do well there, maybe "Which of these two examples of auto-parallelization using Matlab's parfor would fail to parallize the code.." or something like that. Over the course of so many questions, the system would start to paint a more and more reliable picture of where contours of a person's knowledge.

This is time-consuming, of course, but over time it would get easier and faster to find contours by using the 'average' of people with similar backgrounds as a starting point.

Once a fairly good mapping is done of the person to Watson's cognitive model, Watson would need to trace back to the source(s) of nearby concepts and offer them to the user, which, ideally, are then rated by the user for relevance and perceived difficulty to further refine the person's model and rank the material offered for that particular profile.

Now imagine a Grad Student asking a similar question. Or a middle school student. What would those interactions look like? The mappings? The suggestions?

Don't get me wrong...mapping a person's knowledge space is a Very Hard Problem. Watson takes a kitchen sink approach that just isn't possible for a human being. And maybe it wouldn't be possible to tease apart the resulting cognitive model into tidy nodes enough to map anything to. These were questions I'd hoped that IBM could help answer. Instead, it was on to the easy, well-understood problem and solution.

This already exists, it's called adaptive testing. I guess the new part would be modelling somebody's ability, not in one topic, but many related topics.

Sure. Existing adaptive tests were an inspiration for the idea, of course. But putting together a good adaptive test is, itself, very time-consuming. And the test itself doesn't do more than measure ability. The crux of the idea is to use the adaptive test to suggest learning material from a wide swathe of sources to the individual.

Watson is the IBM marketing department going mad about ways in which IBM can continue to remain relevant in a world that increasingly doesn't care about what hardware a particular computer program runs on.

If there is going to be a 'second AI winter' I fully expect Watson and other such efforts to be the cause.

IBM hasn't been about the hardware for a long time, instead it has been about the consulting services contract. And when we, as a startup, first engaged with the Watson folks it was clearly a sales funnel for their consulting services.

That said, IBM has a tremendous amount of research they have done in AI over the years. It is not that they don't have a lot of interesting technology they can throw at different business problems, it seems like they are having a hard time getting invited to the party if they don't track the same hype buzz that the current ML/AI craze has embraced.

The Watson stuff is so oversold it is almost comical.

And yes, sure IBM hasn't been about hardware for a long time, they've been a services company for decades now. But as far as AI/ML is concerned Google and Facebook are attracting the top talent these days, Apple and Microsoft much further down the line.

What would be nice is if they would take the opposite tack, rather than marketing the hell out of it quietly solve lots of problems that are hard to solve in a traditional way. Every time I hear about Watson it is in the context of something where I ask myself "What's the point of being able to do that?". If all there is to hype is the hype itself then it is hollow.

Yes. IBM should be best positioned among AI-aware companies to augment and extend "deep" knowledge-bases into the enterprise. The information infrastructure of Jeopardy Watson was impressive and ought to open doors for IBM to partner productively with other info management vendors to modernize and advance that corporate infrastructure which is driven by deep information. But instead it appears the short-term ROI-think of their non-technical SVPs is what's led them astray.

IBM continues to make the mistake that the flash bang uber-sexiness of ML (esp deep learning) matters more to the enterprise than deep info management (something which IBM can proffer while most competitors can't). If IBM were smart, they'd leverage their deep experience in databases and IR and promote that side of AI -- smarter info management. IMHO, this could do much more for their bottom line than the stupid pretense that Watson really is HAL 9000.

Here I am doing a reasonably good impression of Watson, by totally missing your point and just adding that we have already had (at least) two AI winters, this would be (at least) the third coming up, if it does.

But nothing for IBM to be concerned about. Their marketing has already moved on to the blockchain…

This would be the third or fourth AI winter in my estimation going back to John McCArthy: https://en.wikipedia.org/wiki/AI_winter

I used to work for IBM, but a few years ago.

One thing about Watson that I remember is this presentation by a very senior guy. He had just come back from the US and was presenting what he learned there about Watson Healthcare (IIRC, that's what it was called), which I assumed was a division of the Watson team that was focused on cancer and stuff like that.

I'm paraphrasing, but during the presentation he said something like: "The project was not originally called Watson Healthcare, it was called X (I can't remember exactly), but potential customers were like 'No, no, leave X, we want Watson', so we had to change the name to Watson Healthcare for the sake of our customers. Watson Healthcare actually doesn't have anything to do with Watson."

I couldn't believe, at the time, how much respect I lost for IBM in about 20s. First of all, he thought we're idiots. You have to be brain dead in order to believe that he renamed X to Watson Healthcare in order to help customers. They just wanted to ride the hype train of the Watson brand and were lying to everybody about it.

When your customer is an executive who needs to sell their decision to a board or C-level then pitching "IBM Whatever Health Stuff" is a bit harder than "Watson Healthcare" because of marketing.

his customers are CIO's they need help in selling things to the board, I guess that's what he's talking about.

A lot of CIOs are surprisingly non-technical themselves.

The way IBM talks about it is completely bs. However, this round of AI is definitely better than the last one. Specifically, whats different this time around is that previously, expert based systems and many machine learning techniques require that you specifically hand code things like:

1. Parsing and providing the input dataset into 'features'

2. Hand coding the logic and rules for many different cases (Expert systems)

Now, it has become easier to train a model such as a neural net where you can provide much 'rawer' data; similarly you just provide it a 'goal' in the form of a loss function which it tries to optimize over the dataset.

By 'true' AI, I think most people mean 'how a human learns' - which is actually a very biased thing, since we humans have goals of things like the need to survive, etc. I do believe it would be possible to encode these into goals, although doing that properly and more generically seems a little bit in the future.

One of the neat parts of expert systems was that you could get surprising and bizarre interactions between rules, leading to entertainingly nonsensical answers.

One of the neat parts of neural networks is that you have no idea what rules it's using, but it still manages to produce answers that are sometimes not entertainingly nonsensical.

The bigger problem is that they produce that are most of the time making perfect sense and then every now and then they spectacularly fail on input that is indistinguishable from inputs that gave correct answers.

I think this also happens in the human brain quite a bit. There’s a lot of times where you see something out of the corner of your eye that isn’t there, or you duck because you think something was coming towards you, or you wave because you think you recognize someone.

I bet at a lower level, various systems in the mind fail constantly but we have enough redundant error correction to filter it out.

Well, as referenced in my above comment - you could spend the time to figure out how the prediction was made. Specifically, decoding the training process and what data points were used to influence the prediction - but it would be complicated and take a lot of your time. It might give you more clues why something like random noise was classified as a hot dog.

Ha - the thing to think about here is, pretend that you were asked a question of say, why you labeled a horse picture as a horse.

You might say something like "well, it has hoofs, and a mane, and a long snout, and 4 legs, etc". However, that sort of answer from a computer would probably be labeled as too generic since that could be any number of animals.

The issue is, that to you a horse is some complicated 'average' of all the examples of horses you've seen in your lifetime, that go through a complicated mechanism for you to classify it. Specifically, probabilities of different features that your eyes have extracted from seeing them.

Similarly, understanding why a neural network is doing something is very possible, however, it fails to mean much when the answer it could give is a list of, say, hundreds of vectors and weightings that contributed to its prediction.

It's funny that we are willing to disqualify an AI as an I simply because it comes to some startlingly nonsensical result, yet history is full of humans doing the same thing (from placing a plastic cup on a hot stovetop through to gas chambers).

Feature design is still the prevailing part of machine learning though.

Actually, what makes us human is the lack of fixed goals. As McCarthy wrote circa 1958, one of the properties of human level intelligence is that “everything is improvable”. That means innate knowledge that nothing is ever good enough, and an uneasy tension between settling on an answer and the drive to keep going.

> Now, it has become easier to train a model such as a neural net where you can provide much 'rawer' data; similarly you just provide it a 'goal' in the form of a loss function which it tries to optimize over the dataset.

"just".

Where is the mention of all of decisions on network size, node topologies, regularization, optimization methods, etc.. And you still have to do your first step.

Deep learning is way more complex than just choosing a loss function.

And choosing a good loss function is not a small task either.

> since we humans have goals of things like the need to survive, etc.

What makes you think AI wouldn't have similar ideas about wanting to continue its existence?

Current Ai doesn't have any ideas about existing. That's something we'd have to add in to future AI, which goes to the point of the article and over-exaggerated claims about current AI.

Caring about one's existence (or even having the notion of a self) isn't something that comes about from crunching large data sets. ML isn't going to result in existential feelings or consciousness.

> Current Ai doesn't have any ideas about existing. That's something we'd have to add in to future AI ... isn't something that comes about from crunching large data sets

(note: I'm a roboticist who spends a lot of time thinking about long-term autonomy (how can a robot go for months and years without human intervention) so that gives you a sense of where I'm coming from. Since my original comment was more of a question, I'll expound a little more here.)

I wasn't really asking in the context of current AI, but 'true' AI as the parent was talking about. They drew a distinction between how humans learn and how a "true" AI might learn based on the goals of each, and I question the notion that a "true" AI wouldn't have a notion of survival.

I think you agree with me to some extent, based on your other comment:

> A robot might someday be a self. It would certainly be advantageous for robots to avoid damaging themselves, and being safe around other robots and humans (treating them as selves). But how we go about making robots self-aware, and more problematically, how they have experiences is something nobody knows how to do.

I agree. I believe a robot not only would benefit from the idea of "self", but that it's fundamental to their design. We've built algorithms for a long time that operate in a very limited way. Robots break that mold in a number of ways, foremost being they can move in their world, and even physically change it. Yet we're still insisting on building them the same way we build webapps or trading software.

There needs to be a notion of "inside" and "outside" just to implement things like collision avoidance. I think there will also be notions of "group" and "individual" as well as "self" and "other" and "community" as well if we're going to call robots intelligent, as we see these notions in other species we regard as intelligent (whales, dolphins, chimps, etc).

The question is, how do they get these concepts and what do they look like? I don't know. But I think they're essential, and I think they will be a barrier to AI if we keep treating them optional instead of innate parts of an intelligent creature.

I think we're huge technological leaps and bounds--I mean light-years--away from AGI so advanced it requires some form of computer psychology. But I do think it might eventually happen. And I don't think people are going to do a good job of preparing for it proactively. It's going to require a huge fiasco to motivate people to take that idea seriously.

So I agree that one day we may have to be careful how we treat AGI and maintain a system of ethics for the use of artificial, sensate, sapient beings. But that day is hella not today. Yet when it does come, it will probably still sneak up and surprise us. Like most black swans are wont to do.

Right now IBM is using people's confusion of AI with AGI to make sensationalist ads. But hey, most advertising around trending tech is sensationalist.

What makes you think survival instinct has anything to do with intelligence or awareness?

If anything, I’d guess that the most primitive life forms on this planet have the strongest proclivities for survival.

I think that consciousness is a collection of experiences and observations ('data'), nothing more. I am curious what lead you here:

> Caring about one's existence (or even having the notion of a self) isn't something that comes about from crunching large data sets. ML isn't going to result in existential feelings or consciousness.

My opinion is that 'consciousness' by the human definition will be a spontaneous and emergent property, when the computers become complex enough.

(and also: No, IBM is not there yet, lol!)

> I think that consciousness is a collection of experiences and observations ('data'), nothing more. I am curious what lead you here:

Consciousness is part of having an experience. Observations don't require consciousness, since they can be performed by measuring equipment, although one can get deep in the woods on the whether observation needs a mind to make it meaningful, and thus an observation, not simply a physical interaction.

> I am curious what lead you here:

Being a self isn't about crunching data, it's about having a body and needing to be able to distinguish your body from the environment for survival and reproductive purposes.

An algorithm has no body and thus no reason to be conscious or self-aware. That it's even an algorithm and not just electricity is an interpretive act on our part (related to the deep semantic debate over what counts as an observation).

A robot might someday be a self. It would certainly be advantageous for robots to avoid damaging themselves, and being safe around other robots and humans (treating them as selves). But how we go about making robots self-aware, and more problematically, how they have experiences is something nobody knows how to do. Saying it will emerge once the robot is sophisticated enough is the same as saying nobody knows how to make a machine conscious.

Having had some level of access to inside IBM the whole cognitive initiative has just been this bizarre self-feeding marketing sales escalation where the real engineering has to 'bring the cognitive' in the most Dilbert pointy-haired boss sort of way.

Dan Klein shared in a graduate NLP class at Berkeley a few years back a AAAI article on Watson back in 2010 (when it actually was a distinct technology stack and not just marketing nonsense). At that time IBM was focused on question answering in Jeopardy. It was pretty clearly incremental rather than novel— Dan used the example to show that 1) ensemble techniques can be effective if done properly and 2) hyper parameters matter, a lot 3) there's human intelligence and then there's Ken Jennings intelligence: looking at precision and percent answered, he's in his own separate league. It made me think a lot about individual differences in terms of declarative knowledge.

https://www.aaai.org/Magazine/Watson/watson.php

It was also unclear to me when they did the contest as to whether Watson only had access to the analog audio and/or image of the questions asked. So did they have to parse the question the same as Ken.

Also, it was clearly optimized for a specific use case. If the questions were reworded with more clues that were puns or needed inference, I think Ken would have done about the same but Watson would have faired much more poorly.

They had the text of the question (the sentence), and they had to parse that and then the resulting question was then sent through a text to speech engine obviously, but there was no speech to text.

https://www.ibm.com/blogs/research/2011/01/how-watson-sees-h...

> At exactly the moment that the clue is revealed on the game board, a text is sent electronically to Watson[...]

If anyone is interested, this issue from IBM outlines most parts of Watson in great detail.

http://brenocon.com/watson_special_issue/

"Machine learning" was a pretty good buzz word, but "Artificial Intelligence" is even better. And in a way, ML is part of AI so it isn't really lying.

IBM tries to sell into c-suites of companies that are less technically-adept than the average HN reader. Their marketing seems to be pretty effective, at least in getting proof of concept projects signed with big names.

Watson is simply IBM's ML product, but they call it AI and wrap it in marketing for all the reasons every AI startup does the same thing.

I disagree that companies implementing "AI" are less technically-adept than the average HN reader. This sort of comment has happened on other similar discussions.

It is untrue. They are very technically adept, with teams of people who are also aware of their problem spaces, and technology.

I understand there are some cases of lack of technical teams making these sort of decisions, but it isn't the norm, and even smaller companies often have incredibly technical teams.

I don't understand where this idea comes from. I've done consulting, and implementation of these type of projects for most of my life. My experience says it's false. What is the feeling that this is true?

Is it from people who aren't a part of the process theorizing that some unknown force must not be as intelligent as they are? Is it from looking at the decision making in general (why did they buy an ERP?), and making correlations?

I'm honestly unsure how there's this widespread idea that there aren't brilliant people everywhere doing the same work they are. Yes there are problems, and challenges all over the place, but I find I am amazed all across the country, and world at the level of expertise in companies.

> I disagree that companies implementing "AI" are less technically-adept than the average HN reader.

He didn't say that. He said that the c-suite of those companies was less adept than our fellow readers. This is almost certainly true. I'm not sure that it matters.

Yep, matches my experience. We invited IBM to our company to pitch Watson, there was very little that was impressive about it. "Watson" is mostly just a coding services integration team, who will assign a team to add some basic NLP to your web services. Someone with a free weekend and a book on TensorFlow or NLTK can replicate most of what the IBM sales engineers pitch for Watson.

Oh wow, Roger Schank [1]! Haven't heard that name in a while - he was quite famous in the early days of AI. I wonder if he has figured out a good way to marry ML to his theory of Conceptual Dependency (CD) [2] - because that would could be ground-breaking for hard NLP problems.

Interestingly I started reading the article without paying much attention to who the author is. A few lines in I began to wonder if this is going to be unproductive rant, and if the author has heard of things like CD etc ... It became funny right about then because that's also when I happened to glance at the URL and saw Schanks name.

[1] https://en.wikipedia.org/wiki/Roger_Schank

[2] https://en.wikipedia.org/wiki/Conceptual_dependency_theory

In the 1980s I spent too much time trying to use Conceptual Dependency in a few small R&D projects. Looked promising, but I had little success with it.

I think one of the challenges with using CD in the real world is the "unclean" input that need to be mapped to the various primitives and structures of CD, or stuff in the same vein that came after. Without a way to do that automatically, in a scalable fashion and with minimal human assistance, its utility is limited. Which is why I feel that if we had a way to leverage the current breed of ML techniques to automatically (or even semi-automatically) define this mapping, it would be a big step forward.

Interesting idea!

I'd like to make a plug for my company (http://www.cyc.com) whose "AI" is not machine-learning based, does actual cognition and generalized symbolic reasoning, and lived through the AI winter of the 80s. We've gotten some contracts as a direct result of companies being disenchanted with Watson's capabilities.

I would like to ask you a question: over the years I experimented quite a bit with OpenCyc that you stopped distributing last year. Is ResearchCyc reasonable to experiment with on a small server of powerful laptop? Is an OWL version available?

OWL is not available, but ResearchCyc will definitely run on a laptop

OT from the headline, but I take issue with the author's claim that Bob Dylan's work doesn't relate to the theme "love fades". Dylan has had a vast career beyond his protest song days, and I'd argue that one of his best albums, "Blood on the Tracks" would be accurately summed up as "love fades".

I agree; OP is one of those articles, that start to make a reasonable point only to frame it with a highly debatable specific example.

Dylan didn't win the Nobel for his songs about faded love, and he won't be remembered for them especially.

Schank is 72 years old. He came to know Dylan when when his protest lyrics gave voice to 20-somethings like Schank in the 60's. Like the rest of us, Schank likely remembers Dylan less for his retrospective years thereafter.

What you've called Dylan's "retrospective" years include an earth-shattering comeback with three top-charting albums in a row starting with 1997's "Time Out Of Mind" (which, incidentally, has quite a few 'love fades' lyrics). Dylan himself has singled out "Time Out Of Mind" as the only one of his own records that he himself goes back and listens to.

Thank you. I don't know much about AI, but I like to think I know a little about Bob Dylan. The bittersweet love song is a Bob Dylan staple that lasted well beyond his protest phase. I think Watson actually a pretty good job of identifying it as a theme. From "Girl from North Country" to "Don't Think Twice, It's Alright" to "It Aint Me Babe" to "Tangled Up in Blue" to all the way into the 90's with "Make You Feel My Love", it's a theme that spans his entire body of work.

I also think that even with advanced cognitive computing it would be difficult for a computer to interpret those songs meaningfully without having lived through those times. I don't think a competent human could interpret those songs without huge reams of context. Which is probably the point the author is making about counting words vs. cognition.

This isn't a slight against Dylan, he's amazing, but perhaps it's an unfair example to judge an AI by how it figures out those lyrics. There's some very intentionally abstract content, some inspired by cut-up poetry of the beat movement which literally uses randomized words to induce pareidolia and free-association. Even with cognition, you can't correctly interpret those ideas--you form your own unique and subjective perception of them.

Would we even be able to tell if we invented a computer with proper appreciation for absurdity?

But on the subject of IBM: yes, they are making sensationalist ads.

Dear engineers, merit is useless when you're trying to sell something. Authority is king, and people remember emotion and hyperbole. Marketing and sales is almost always about representing authority regardless of merit. The only thing that matters after a Watson sale is if Watson can help solve the problems the customers have.

True, it's well known that engineers don't often have appreciation for the emotionality of marketing. But there's a limit here. We can't just have no-holds-barred hyperbole and outright lying to the point of unfairly deceiving customers.

Obviously it takes a legal professional to judge where that line falls, but for something as specialized as this it's hard for laypeople to appreciate the distinction between stretching the truth with enthusiastic self-promotion and full-on false advertising. It's interesting to think about.

Well, it might be argued that with the exception of this author (who apparently coined the terms they're using) neither the IBM marketers, nor any of the potential consumers understand what these words mean. It is pure abstraction to most people. If the marketers were pressed, I'd guess that they'd just define these terms differently than the people making claims of fraud.

I'm imagining the glossy eyes of a judge or jury trying to grasp the nuance and just throwing in the towel.

Well, there's also the opportunity cost of capex leaked away from areas that it could be productively focused on, also the opex of running the new implementation. And the missed opportunity from building skills in implementing opensource alts - like Tensorflow, for example.

Since we're thinking like CTOs we need to account for risk, not cost. IBM sells Watson to companies with money, and for those clients the factors driving decisions are more often risk and liability. A CTO with budget who chooses to build in-house is taking a big risk. If IBM fails, they can cover their ass with a contract. If they fail in-house there is no such safety net. Easier to save face with the board.

In other words, nobody ever got fired for buying IBM

Anyway none of the company that pretend doing AI are actually doing AI. AI nowadays is pure branding bullshit.

I heard someone say that A.I. is just what we call technology that doesn't work yet. Once it works, we give it a specific name, like "natural speech recognition".

However, if a robot from scifi were to walk out of the lab, like Data or Ava from Ex Machina, or we had access to HAL or Samantha from Her, we wouldn't just give it a specific technical name. We would consider those to be genuine AIs, in that they exhibit human-level cognitive abilities in the generalized sense.

It's true that in Her, Samantha was just an OS at the start, kind of like how the holographic doctor was just a hologram at the beginning of Voyager, but as both stories progress, it becomes clear they are more than that. By the end of Her, Samntha and the other OSes have clearly surpassed human intelligence.

Those are fictional examples, but they illustrate what we would consider to be genuine artificial intelligence and not just NLP or ML. The reason people always downplay current AI is because it's always limited and narrow, and not on the level of human general intelligence, like fictional AIs are.

I like that definition too (I know it from Seth Godin). It’s honest, in the sense that we just don’t know yet how to that stuff instead of labeling every single code of line as AI.

I think a reason for this is that in the early days of computing and AI research, strong AI / artificial general intelligence (AI possessing equivalent cognitive abilities to humans) was considered both to be within reach, and the most obvious solution to many problem domains. We now realise that things such as computer vision and natural language translation can be approximated with solutions falling far short of strong AI.

Personally I define AI as software that you "train" rather than "program". In the sense that neural nets and other ML tools function as black boxes rather than explicit logic.

By that definition, AI is a real thing—it's built on top of programming that uses compilers and languages and ones and zeros—but it's different and it's valuable.

To say it's all bullshit, I feel, is to cut yourself off from new skills. Kind of like "compilers are all bullshit—it's opcodes at the bottom anyway."

AI carries a set of connotations in popular imagination that a) don't comport with the actual capabilities of what we term 'AI' in the computer science world, and b) are being exploited by marketing teams at IBM and plenty of other companies to sell technologies that aren't particularly new or interesting. The kernel of truth in 'AI is bullshit' is really that the discourse around AI is bullshit, which I think is a pretty fair assessment, and this is coming from someone who's work gets labeled as AI on a regular basis.

>I define AI as software that you "train" rather than "program".

I like this definition. It covers things that are AI but not ML, like DSS / rules engines. I've built two fairly sophisticated DSS before but haven't messed with ML much. It seems interesting, but I haven't had the time.

https://en.wikipedia.org/wiki/Decision_support_system

Eliza is the first AI program I came in contact with on the Commodore. It was built in the 60s.

https://en.wikipedia.org/wiki/ELIZA

AI is a very broad subject and ML is just a particular (promising) technique to perform AI.

Not sure what you mean by that. Is there some industry standard around the term "Artificial Intelligence"? I agree that its become a bit of a buzzword, but I'm not sure that its being misused.

When I hear AI I usually imagine deep learning, but many companies using the term don't specify.

But that's more "machine learning" which always seemed less "sexy" than AI -- basically just regression but better, not magical like AI.

I will say that when my company uses AI, they almost always just mean LSTM-based content generators - usually chatbots or "advisory"-style outputs - but the key idea is that it's generative and not just an evaluator.

I think that's probably the most helpful definition because your ML output has to go into some larger intelligence system (human or otherwise) to produce some decision / activity. So your choices are:

* Human * Expert system with rules of interpretation that include ML output as input * AI system which relies solely on inference and reinforcement / goal-seeking to produce output

"This was about promoting expert systems. Where are they now?"

In 2017, Intuit, Inc., owners of Quicken, posted revenue of about $5 billion.

Not too shabby.

As someone who is currently working on successful commercial product with a strong a expert system component, I agree with the sentiment. The funny thing about this project is that the product owners nor marketers and not even the coders ever use the term "expert system". It just doesn't sell any licenses nor garner any attention.

My view is that expert systems are a ubiquitous part of many products to the degree it's hard to even recognize them as such. They're not the main focus of anyone's marketing budget, because that makes about as much sense as promoting your "revolutionary axle technology" to sell a car.

A bit off topic, but I've been wondering about what expert systems are a lot lately, and I hope you don't mind me asking a few things about them.

I started studying machine learning long after statistical models was the absolute standard, and all I really know about expert system, are passing phrases in textbooks about how the world has moved away from it.

How does one go about building one? MNIST character recognition is often called the "Hello world of machine learning" ... what is the "Hello world of expert systems?"

Is there a modern term for Expert Systems?

the term 'expert system' was tainted by AI Winter I.

I suspect the term 'Machine Learning' will become a dirty word after AI Winter II.

Heh. Of course that’s a space with a pretty constrained and well defined set of rules. Until they aren’t of course. Which is why we still have accountants.

It's a pretty open secret in the community that what IBM pitches that Watson can do, versus what it (or any state of the art system) really does is pretty much bunk. This author calls it fraud, but a more charitable interpretation would be extreme marketing. We've seen a lot of failures with Watson, particularly in the medical space - MD Anderson's Cancer work for example (https://www.forbes.com/sites/matthewherper/2017/02/19/md-and...) where MD Anderson payed around $40 million (on a original contract deal somewhere near $4 million) and eventually abandoned it.

I do think Watson may be a fake it until you make it thing - in particular, they still have access to a incredible amount of data, and data determines destiny on a lot of AI.

but their message is for outside the community, so I don't think they deserve any charitable interpretation, they lost this privilege a long time ago. I have to say I'm biased, I wish I can see them disappear soon, but I think I have the right motivations.

Had IBM sales people present on Watson as a security solution recently. The stench of BS was so bad I nearly had to leave the room. It wouldn't bother me if they kept things generic, but they deliberately sprinkle the presentations with specific terms referencing hyped technology (deep learning, etc), with the clear objective of deceiving the audience into thinking they are using those technologies when they clearly aren't. It was unethical IMHO.

Did your company still buy it?

No, we aren't quite the target market yet, but I can see the ground is being prepared for when we might be.

Chomsky calls it a sophisticated form of computational behaviorism. Just like the research program of behaviorism died out, this will eventually too. There are other respectable criticisms of AI, like Hubert Dreyfus' 'What computers can't do'.

Neither Chomsky nor Dreyfus claimed that machine learning and/or AI won't solve any problems, but rather that the kind of problems these solve are not relevant in terms of aspiring to be humans.

Chomsky's "Where AI went wrong" is often ignored by the mainstream AI community or dismissed. Peter Norvig's retort was poorly constructed, and showed that he didn't comprehend Chomsky's argument.

Machine Learning and AI are stuck in a rut and apparently these so called ML/AI experts know better. The Deep Learning (along with ML) craze is a hindrance to a true scientific theory of intelligence.

I'm no expert in the field of AI or Machine Learning, so I have a question for the experts here? Has there been any theoretical breakthrough in the AI in the recent years? I knew we had neural networks and different types of classifiers, etc. for more than a few of decades now, so apart from better marketing, has there been a significant breakthrough that explains this sudden surge in the interest in AI?

There have been a number of breakthroughs in specific fields recently. In fact it seems like there is a paper every week that pushes the state of the art forward in some branch of AI/ML. I think the big one that triggered the current excitement was the success of convolutional networks in image recognition tasks. You can read about that one (in 2012) here: https://www.technologyreview.com/s/530561/the-revolutionary-...

EDIT: This paper refers to the algorithm as SuperVision which was the team name, but it is more commonly called AlexNet. Here is another article discussing it:

https://qz.com/1034972/the-data-that-changed-the-direction-o...

Thank you. The very answer I was looking for.

Before 2005 experts estimated that AIs that could beat humans at Go would take many years still. In October AlphaGo came out and in March 2016 beat everyone. In 2017 AlphaGo Zero came out, which was only trained on self-play. It beats all previous versions.

But it is not so much these big breakthroughs that cause current optimism, as much as the large amounts of continuous improvements and refinements.

Recent breakthroughs in ML from a commercial perspective are achieving super-human performance in some computer vision tasks and end-to-end methods beating traditional ones in machine translation and speech recognition.

However, these breakthroughs have not much to do with a general notion of intelligence as they are very limited to a specific task. In the media, the term AI is used very inflationary these days.

> has there been a significant breakthrough that explains this sudden surge in the interest in AI?

Yes. Some tasks like language translation, face recognition etc. have reached human level performance. Why has this happened? Large data and increase in computing power (Thanks gamers and GPUs!).

Is this translation service available online? I tried different services recently while travelling, none of them were better than basic knowledge of the language and a dictionary. Full sentences sometimes work well but anything else is problematic (e.g. restaurant menus).

deepl.com is pretty good. This translation of your question into German is flawless: "Ist dieser Übersetzungsservice online verfügbar? Ich habe in letzter Zeit auf Reisen verschiedene Dienste ausprobiert, keiner von ihnen war besser als Grundkenntnisse der Sprache und ein Wörterbuch. Ganze Sätze funktionieren manchmal gut, aber alles andere ist problematisch (z.B. Restaurantmenüs)."

Some deep net answers below.

However - MCMC for scalable bayesian reasoning. - Answer Sets for reasoning in FOL. - Big data sets for training lots of classifiers. Including deep networks. A big breakthrough has been crowdsourcing the labelling.

I saw a demo of Watson a couple years ago at a trade show and was not super impressed. Looked like a glorified Markov chain to me.

It’s an unusually beautiful written article, well worth the read just for the prose. As for the main sentiment that we have a new AI winter, I’m not so sure. My lay person view is that we see quite a lot of commercial success with these systems so the current wave will be well funded for at least a decade.

What are the big successes? Speech recognition? Which still seems rather bad to me.

Language translation (which for the languages I’m interested in (Japanese) is still almost totally unusable?

Self driving cars? Which are not yet in production (and where the social issues are probably far harder than the technical ones, and likely have been since the 90s).

Is there some big application if ML that I’m missing that is a clear win?

A lot of ML behind the scenes. Search, knowledge systems, map routing, etc. You know, the boring stuff that isn’t AI yet. I don’t really disagree with your broader point. I expect a lot of things many think are just around the corner are going to be many years away.

Amazon's system for finding related products is eerily accurate. The more you buy, the better it gets at anticipating what you want to buy.

... but the larger wins they've had are behind the scenes, in the predictive modeling that helps them get product into warehouses in front of demand spikes (and into geographically-relevant warehouses to satisfy the spikes).

> Amazon's system for finding related products is eerily accurate.

Amazon's system for finding related products fucking sucks. After I buy an X, I see ads for more X for months, which is completely useless. Sometimes I'll see ads for the exact thing I bought... what?!

I'm not sure that things like Amazon's "Customers who bought this also bought that" is particularly amazing or clever; that ain't rocket surgery, you know! Sure, every once in a while I'll go "Yep, I probably need me some of that, too", but mostly it's "Nope - I've already got plenty of that, thank you. Moving on ..." And like everybody else, I too get sick of being pitched even more of this, or different variations of this, after I've already made my purchase. I'm sure that all of it does lead to at least an incremental increase in sales, though, so it serves its purpose.

A while back I saw an article on the success of an "A.I." that was being used for marketing purposes. And as I read through all of its supposed "breakthrough suggestions" I thought to myself "You know, you could have saved yourself an awful lot of time and trouble and money if you'd just read pretty much any sales and marketing book from, say, the last 100 years or so." Because there was really nothing there that wasn't already pretty much common knowledge within that realm.

I work on Amelia at IPSoft, things seem to be going well.

Customer Service is a huge, overlooked industry mostly based around classifying requests and then performing a rote action, which is a great use case for AI.

I used to work at IPsoft as well. I think the big difference is between "ML as a tool to solve a 80%+ of relatively standard customer and employee service requests" vs "we've created self-reasoning AI that will solve all your problems".

In my opinion, sales at IPsoft faced some of the same challenges initially (selling the idea of AI vs selling a well working solution for a specific problem), but has become more focused now that they have proven use cases in the field.

As you say, service desks are a great opportunity for ML. They're seen as cost centers, typically plagued by high attrition of employees, provide inconsistent service levels, and spend the majority of their time on requests that are relatively standard. Still, it is not an automatic effort: integrating with back-end systems, creating a dataset to start classifying incoming requests, and defining the processes to handle them are (mostly) still manual work.

Not sure what you are using, but speech recognition is amazing on my android device and has been for years. I rarely ever type anything into my phone anymore. It's a usability gamechanger.

ML is a "big" application at the tools level. Categorization in particular.

You're already seeing it in product. Consumer level security video equipment with human detection is pretty accessible now. I can ask my iPhone for pictures of my kids in snow in 2014 and get a pretty good output. Enterprise level categorization of photo and video is a thing.

"Smarter" machines are just like smart people, by themselves not very exciting. But give them a purpose or application, and things get exciting.

> Self driving cars?

For safety reasons, self-driving cars on the road don't rely on ML as much as you may think. It's much better to run algorithms with provable guarantees rather than black-box neural nets on a multi-ton vehicle. When the judge asks your company if you did everything in your power to prevent the death at hand, it helps to point to decades-old well proven techniques, rather than band new unproven, untested, trends.

ML is the new A/B testing

> Self driving cars? Which are not yet in production

Yes they are. What do you mean, not in production? Maybe there aren't millions of fully autonomous cars on the road today, but there will be soon. They sure are in production. Companies like Tesla are making production hardware self-driving cars right now - full autonomy coming soon, of course.

> Speech recognition? Which still seems rather bad to me.

Speech recognition is 100% amazing. The phones never ever hear me wrong any more, not ever, not once. The main problem I have with it is interpretation, which can be shocking, shocking bad. (Ask Google for a joke, it'll tell you one and then prompt to you ask "one more". As soon as you do, it'll say "one more what? I don't understand", which is pretty funny.)

> Is there some big application if ML that I’m missing that is a clear win?

Well yeah. I would say millions. ML has been embedded into the world now and makes nearly every daily interaction with technology better than it was before.

> Yes they are. What do you mean, not in production? Maybe there aren't millions of fully autonomous cars on the road today, but there will be soon. They sure are in production.

Huh? They are in production but not yet in production, but they sure are in production? Can I go to the store and buy one? No. So they are still building the things!

> Speech recognition is 100% amazing. The phones never ever hear me wrong any more, not ever, not once. The main problem I have with it is interpretation, which can be shocking, shocking bad.

I simply do not believe your first point, based on my own experience. 100%? Really? Moreover, your second point, that the interpretation is off, is most of the problem. That's where 99% of the work for the foreseeable future will be spent on this problem.

Self driving cars that can dependably not run into things or people do not exist.

Perhaps it is beautifully written but unusual? It seems pretty baseline to me. Certainly doesn't seem to meet the bar of worth reading, regardless of its argument. I mean if you only read twitter all day, then ok.

I feel like they've moved on from their lies about Watson's capabilities to lies about their capabilities with blockchain technology.

> People learn from conversation and Google can’t have one. It can pretend to have one using Siri but really those conversations tend to get tiresome when you are past asking about where to eat.

At first blush he sounds like my technologically semi-literate grandma who would definitely conflate Siri and Google as being part of the same grand internet program. I had to read this twice to understand that in saying "it can pretend to have one using Siri", he meant that asking Siri a question sometimes redirects to a Google search, but wrote it in a way that personified Google as the actor with intent in that transaction. What an odd and paradigm-breaking way to look at that.

The problem is that even internally, there is this notion of "sprinkling a little Watson to do the hard jobs" and then 'poof': problem solved. Marketing reflects internally, too.

Can an mod add the date to the title? This piece is from 2015.

AI is definitely starting to enter the “through of disillusionment” in its hype cycle.

Is it really? I don't see that at all. Rather I see AI as finally being entrenched in normal, everyday products and services. AI is here to stay and the hype hasn't even started yet, at least not compared to what's coming.

Billions of people own devices that they can talk to, that can talk back, that can translate between more languages than humans can, that know facts about you that you didn't even know yourself, etc.

Basic AI has come to be an expectation of many consumer products now. And real AI is coming faster than ever.

Fake audio and fake video, generated from computers/AI is here. Self-driving cars may be just a domain-specific expertise, but it is still AI by any traditional definition.

AI is not reaching any kind of trough of disillusionment that I can tell. We're still obviously just getting started with what can be done.

</AI hype post.>

While there are domains where AI is having successes, the current general expectation is far ahead of where the technology is. People think (and in part due to the marketing like IBM is putting out) that ML/AI can do things that aren't possible yet.

There may be a reset in expectation soon which will lead to become more pessimistic at claims being made and the marketing. Once we are through that, it will be easier to get people to understand what use cases align with the available technology and implementing ML/AI will become more productive. Aka the Trough of Disillusionment followed by the plateau of productivity.

Yeah, I guess you’re right, too :-) It’s definitely true for ML. However, I was mainly thinking about the overhyped “AI can do anything” sort of nonsense that is mostly spread by people who don’t have any clue about CS

oh man you can say that again. Look ma i can perform linear regression on a bunch of data , i'm a data scientist!!! Even better i just heard a new term 'IA', local tech meetup has a presenter with headline

"discussing an Intelligence Augmentation (IA) approach to human-assisted artificial intelligence."

oh god now we have IA and AI. holy shit!

The real time voice translation Microsoft is doing in Skype is genuinely impressive. Pity the rest of the app is so shonky!

I disagree. I read this article a few years ago. It is true about IBM, but other companies have significantly more interesting and useful applications for ML right now.

I would only refer to Episode 4-5 of Silicon Valley in Season 5.

This will be the mistake we will make when we introduce a technology we think it is smart enough to make decision for us and turns out all it does is read words faster than us and interpret them literally.

Even with the closing statement of "AI winter is coming soon." I can see Watson having a problem understanding that statement even with context.

Something feels wrong about the assessment of the second block of text being written by a human.

I feel some conflict in my head that someone would talk about Bob Dylan as "overstepping" a claim about his prominence and then conceding that he does belong in a "Top 10 Bob Dylan Protest Songs list." Of course he belongs in a Bob Dylan list he is Bob Dylan.

Does that sound human?

I think the text is pulled from a "Top 10 Bob Dylan Protest Songs Lost" article.

Watson's marketing is obnoxious, but it's not just Watson. There is plenty of bullshit, ignorance, and pseudo-intellectualism to go around. Mind you, many of the technical fruits of AI itself, properly understood, are not bullshit (the name "AI" is misleading IMO; I wouldn't be able to tell you what distinguishes AI from non-AI because it seems largely a matter of convention rather than a substantive difference). The field offers plenty of useful techniques for mechanizing things people have had to do preiously. However, the very idea of a "thinking computer" is unjustifiable and superstitious. There's too much sloppy, superficial thinking.

The author of the article mentions concepts and indicates a distinction between them and word counting. Certainly, there is a difference between word counting and conceptualization, and it is patently obvious computers don't do the latter. But it's worse than that. Technically, computers aren't even counting words. They aren't even counting, nor do they have any concept of a word (we count words by first knowing what it is that we should be counting, i.e., words). What we call word counting when a computer does it is a process which produces, only incidentally, a final machine configuation that, if read by a human being, corresponds to a number. The algorithm is a proxy for actual counting. It is a process produced by thinking humans to produce an effect that can consistently be interpreted as the number of words (tokens) in a string. That's not thinking. There is zero semantic content and zero comprehension in that process, and no number of tortured metaphors or twisted definitions can change that. AI, as it becomes more sophisticated, is at best a composition of processes of the same essential nature. No degree of composition -- no matter how sophisticated or complex -- magically produces thought anymore than taking sums of ever more composed and expansive sequences of integers ever gets you the square root of two. It's not a mystery.

IBM's customers probably understand that Watson (the jeopardy-playing bot) isn't really relevant, and that what they're buying isn't a pre-written software package so much as software consulting services. But there's still a serious problem, which is that a customer of Watson would reasonably believe that they're getting the team of engineers that solved Jeopardy. In reality there is no overlap whatsoever in personnel between Watson-the-PR-project and Watson-the-thing-you-hire.

Watson Services are very much a product you buy/subscribe to. Just like the Machine Learning services you get from Microsoft or Google. You can use them pretty easily with absolutely no consulting at all.

I always feel that this is one step away from that famous quote, "The greatest trick the Devil ever pulled was convincing the world he didn’t exist," attributed to various (https://quoteinvestigator.com/2018/03/20/devil/). In this case, the greatest trick is convincing that it does exist... and maybe is the harder one.

I don't have gripes with this marketing approach and I read it more as "expect to be surprised by what is possible" rather than harp on what isn't .. at least not yet. For a comparison, it is like apple branding their display tech as "retina display" to communicate the intention (you can't tell pixels apart), possibilities (you can now use any font) and quality rather than any claims about mimicking the eye.

Start by asking "What is Watson for?"

Watson is for helping decisions at large corporate customers: The CEO has heard of Watson. He saw it on a screen in the VIP tent at the golf tournament, and thinks it's neat. The CIO feels safe with "Watson" in an RFP response from IBM because the CEO thinks it's neat. IBM is happy with this pettifoggery because it keeps the SOW vague and open to maximizing revenue from the project.

It's not about AI.

“These guys are a fraud. Come look a thing my online Academy. Call for the price.”

Roger Schank used to be a serious researcher. Also tied to consulting firms like Accenture.

I think the biggest concern is that the general public outside of this industry think that AI and machine learning is Hey Google not understanding them and their bank working out you’ve run out of money – which they already know. When AI _actually_ arrives, they’ll be bored, ignore it, and then, well, I guess it’ll know and take over the world. We’re all doomed.

>A point of view helps too. What is Watson’s view on ISIS for example?

>Dumb question? Actual thinking entities have a point of view about ISIS. Dogs don’t but Watson isn't as smart as a dog either. (The dog knows how to get my attention for example.)

Ouch. But at the same time, for something that didn't grab his attention he sure had a lot of words to say about it.

>...counting words, which is what data analytics and machine learning are really all about

The piece is welcome anti-hype, but, what? How can a true expert in the field say something like this? Or, maybe I should tell my colleague who is working on ML for diagnostic radiology to think of voxels as, uh, words?

> I started a company called Cognitive Systems in 1981.

Ahh. So just from the article, his gripe is of the “begs the question” sort — he’s not pleased with the evolution of idiom. Since he was doing “real AI” back then, who are these frauds to claim they are doing AI?

His point may or may not be valid, but his specific argument is quite weak. He notes that even a person, an actual intelligence, wouldn’t know what Dylan was singing about without context. He goes on to presume that Watson doesn’t have context, but who’s to say? Watson could certainly read all the articles about Dylan that he so helpfully cites, and come to “understand” the songs. And maybe Watson has.

If you follow the links to his academy, you divine a bit more of the motivation.

His software development course provides “a unique automated mentor, employing natural-language processing technology derived from our decades of artificial intelligence research”. He is desperate to stay away from calling it actual AI, yet can’t resist implying that it is. This is probably most irksome to him.

My advice: sometimes you have to join ‘em.

He specifically addresses that he _wasn’t_ doing “real AI” then, and given they’re doing fundamentally the same or similar things, IBM aren’t now either.

Yes, that is my point (sorry that I mis-stated it a bit). He is offended by the claims of these charlatans when he knows the technology hasn't advanced to the point of being AI. How dare they claim otherwise.

Also, he knows that they know it's a lie. Unforgivable.

But he's fixated on the literal (once-true) meaning of "AI". It doesn't mean that anymore, not to the lay person and not to the average technologist either.

Like "begs the question", one has to just get over it.

Not to the lay person? Really? Do you have any evidence to back it up? Hint: HN is not full of lay persons.

I remember seeing "Watson" mentioned in the news like five years ago, but besides a couple HN threads, I've not seen it mentioned since then. Am I missing something (besides TV, which I don't watch)?

Here's a great video that covers "AI winters" for those who are curious: https://vimeo.com/170189199

Isn't it more or less common knowledge now that Watson is all marketing buzz? The Watson that IBM is selling CIOs on is a very different thing from what was seen on Jeopardy.

Discussed at the time: https://news.ycombinator.com/item?id=11751267.

We won't know how to build machines that understand until we know what understanding actually is at a biological level. I'm not convinced that we do.

I am not even convinced that understanding happens at a biological level. I think there is still a lot that can be done to model aspects of human reasoning in software to produce useful results, especially if it is married with machine learning, but I don't think we will get there by looking at biology.

Machine Learning in a lot of ways is legitimate, but its applications to a lot of NLP / chat-bot technologies is far from cognitive computing.

"People learn from conversation and Google can’t have one. It can pretend to have one using Siri"... Google doesn't use Siri.

"Search is all well and good when we are counting words, which is what data analytics and machine learning are really all about."

There are machine learning models which go far beyond counting words, for example see https://arxiv.org/abs/1502.01710

Parent links to Yann LeCunns's "Text understanding from Scratch" paper, from 2015, where the authors uses a conv-net, originally build for image recognition to do text categorisation.

The NN techniques falls squarely in the "counting words"-bracket, although this one is actually counting characters.

It is a great paper, with great results, but none of those models therein have an opinion on ISIS, an ability to converse or anything the author of TFA calls cognition.

I do agree that Watson seems oversold, but the evidence in this article (Watson's shallow opinion of Bob Dylan, compared to the author's opinion of Bob Dylan) seems kind of weak. I was hoping for some insider information on the implementation of Watson, but unfortunately there is none.

The day AI understands my dirty jokes, it's the day I call it cognitive.

It's just advertising.

False advertising. Is that not worth calling out?

(2016)

that's just auto generated... I saved the bookmark in 2016... could be even earlier. I don't see any dates. Wayback machine has its first saved copy in May 2016 which matches my bookmark.

http://web.archive.org/web/20160523070729/http://www.rogersc...

Methinks voice will in fact win.

Normally my comments are very sceptic, but this article is just spot on.

The problem is not only context or subtext, it is even worse from an entirely predictable standpoint:

Consider a large corpus of text (books, articles, ...).

1) Concepts that are WELL UNDERSTOOD BY HUMANS will not be explained by humans when they reference them: the word or concept "pet" (as a verb) will show up in many sentences refering to the petting of cats, dogs, horses,... and ML will correctly predict the conjunction of "pet" with any of these words in the sentence. It will even be able to train ML to confabulate realistic sentences in the sense of echolalia. Consider then the sentence:

"The lady pets the cat"

The computer will recognize that the presence of cat is not surprising (bingo!). The computer will have no idea that this probably involves one or more cycles of the lady's hand gently pressing down on the fur belonging to the cat, then while still preessing down, moving the hand in the 'natural direction of the hairs' (NOT the other way around) probably from closer to the head towards the tail, and probably lifting the hand before repeating the cycle so as to massage the cat or perhaps so as to remind the cat of its time as a kitten being licked by the mother cat.

No book or conversation in the corpus will give this detailed description exactly because humans expect each other to understand this.

2) Concepts that are POORLY UNDERSTOOD BY HUMANS will be vigorously (but often erroneously) explained by humans communicating to each other what they think is going on: endless texts about religion, sexuality, perpetuum mobiles, economy, ...

How do we even expect the computer to produce a sane result, even if it correctly guesses the context?

That said, I do believe relatively helpful natural language processors to be possible, but they will have to be vigorously trained by multiple human curators individually analyzing a sentence and trying to find (probably true) statements about what a sentence implies:

Starting again with:

"The lady pets the cat"

One curator might mention hands touching fur while moving.

Another curator might add that one can also conclude the hand probably presses down on the fur.

Yet another notes the sentence also implies the lady is still alive, for else she would not be able to pet.

The first curator now adds that the cat as well is probably alive, for else the lady would probably not want to pet the cat, since massaging is useless to a dead cat.

The third curator now mentions that the sentence implies one or more cycles of an individual pet stroke.

Etc...

As you can see this quickly becomes an expensive operation.

Now one might train an adversarial neural network to look at a sentence (or sentence with context) and a list of probable conclusions, to predict if the list of valid conclusions is complete or incomplete. And then only send the incomplete ones to humans?

What is the point of changing the headline of an article like this? The headline was an accurate summary of the contents ("THE FRAUDULENT CLAIMS MADE BY IBM ABOUT WATSON AND AI"). Maybe you agree, maybe you don't. But the current headline is just wrong.

From the article:

> I will say it clearly: Watson is a fraud. I am not saying that it can’t crunch words, and there may well be value in that to some people. But the ads are fraudulent.

That's what this is about. Not "Claims made by IBM about Watson and AI."

I changed it in a bit of a rush earlier. Note the word "unless" in the site guideline: Please use the original title, unless it is misleading or linkbait. "Fraudulent" in the title was guaranteed to provoke objections—you might not consider it misleading/linkbait but other readers would.

Normally we look for more accurate and neutral language in the article text to use as a replacement title. Now that I'm not in a rush, it's clear that the subtitle contains such language, so we'll use that instead.

Edit: funnily enough, that's exactly what we did with the title when it was posted two years ago: https://news.ycombinator.com/item?id=11751267.

The HN submission guidelines include the following rule:

"Please use the original title, unless it is misleading or linkbait."

The original headline is a bit sensational/clickbaity. That's a judgment call in this case, IMO, but I suspect that's why the title was changed.

I wish click bait was more well defined. In this case, I feel the original title summarizes the entire claim made in the article. Using the title "Claims made by..." sounds like it'll be an article summarizing the claims made by AI and Watson, rather than a push back on those claims.

I'll go ahead and define clickbait using a simplified and possibly anthropomorphized version of information theory.

First:

- The more improbable a message is, the more information it contains, assuming the message is true

- If the message is untrue it contains no information

- If a message is already known by everyone it contains no information

Not Clickbait:

If a message is surprising (seemingly unlikely or previously unknown), and true, then it contains high information, and will be very likely to be clicked. This is not just a good thing, it's the most optimal thing!

Clickbait:

If a message is surprising and untrue, it will also very likely be clicked if the user cannot easily determine that the message is untrue. The user may then be disappointed when they discover the message actually contained no information because it was false. Incidentally, false messages will always have a high probability of appearing to be high information messages, because they will often appear to have the least likelihood.

Incidentally, this is (in my opinion) the theoretical problem of fake news. It will always appear to be high information to those unable to determine if it's true or false. In other words, it will appear to be of the highest value, when really it has no value (or even negative value if you look at the system level rather than just information level).

I disagree, and do not believe the title is linkbaity/sensational at all, and find it problematic that the title was changed. It is an accurate description of the author’s position, which is that the claims made by IBM are fraudulent. Holding a position isn’t linkbaity or sensationalism.

A linkbaity or sensational title would have been “you won’t believe what IBM claims”.

Exactly this. I've been uncomfortable with this trend for a while now, where everything gets labeled "clickbait" and there's a thread of resentment if an article does anything other than lay out dry facts as efficiently as possible.

The pendulum has swung too far. This wasn't clickbait, it was a fair and accurate summary of the full piece. Flag the entire post if you think it's inherently inflammatory, downvote, ban it if you're an admin, but this title had no business being edited.

I'm sure you're right about the why, but I don't believe this was a judgement call, I believe it was wrong. The current headline is misleading (if the purpose of a headline is to provide information about the contents of the full post).

Questions about whether the entire article is misleading should be addressed via discussion in the comments or, if it's egregiously wrong, flagged/downvoted/removed.

> The original headline is a bit sensational/clickbaity.

Oh, so having an actual opinion about something is clickbaity now?

You should the CEO’s of big companies bragging with big buzz words like “AI”, “cognitive”.

The truth is very few people understand the current state of AI and it’s limits. Most of the fancy papers in Arxiv are based on NN’s which are just essentially vector algebraic math.

Yes, we made huge leaps in image labeling and speech to text but current AI is a far cry from the full abilities of human brain reasoning and comprehension.

The reality is people will say whatever to make a sale / get VC funding.

The current economy is running on greed. When the bubble bursts, people will get hurt. Then the economy runs on fear. Fear that AI is hype.

IBM has always been about hype. They are riding on the jeopardy wave and exploiting the greed.

You can also argue that even the leaps we've managed to make in machine learning (a phrase I prefer over 'AI' not least because it includes the reminder-word 'machine') are mostly quantitative rather than qualitative leaps. Almost anything you can think of that's 'better' lately seems to be mostly thanks to 'more' -- more data collected in one place available for training/processing (thanks to large centralized surveillance companies), and the ability to process more data faster (thanks to faster hardware, and more of it, owned by those selfsame large centralized surveillance companies). We've seen a lot of optimizations and interesting research within the existing paradigm but we haven't seen a serious qualitative breakthrough the likes of which would justify this kind of hype.

"just essentially vector algebraic math" -- True, but only in the sense that anything done by a computer is just iterated Boolean logic. Yet, Wikipedia and Grand Theft Auto and Mathematica have emergent properties beyond the obvious things that Boolean logic can do. So the fact that neural networks are based on vector algebra doesn't prove that they aren't capable of impressive things.

I mean to say the there are tons of problems NNs aren’t good for. If we are on the journey to discover GAI super algorithm that is smart enough to learn very quickly without a shit ton of data, I’d say we still have a long way to go.

Anyone claiming they solved cognitive reasoning with a bunch of NNs stacked together in a black box is probably lying to themselves.

Everything IBM says about Watson should be taken with a few tons of salt.

I don't know how much I'm allowed to say, but they couldn't even get it to work acceptably internally for some of the basic datamining and natural language processing that are among the things so highly touted in some of their TV advertising. This is with a gargantuan dataset compiled from years of relevant interactions to train on in the particular area of interest.

Re: Everything IBM says about Watson should be taken with a few tons of salt.

How is this different from nearly all other "AI" companies? IBM is not the only one guilty of hype. Neural nets can do specific things well, such as mirror a specific training set well, but still have major gaps in terms of what many call "common sense". It does smell like AI Bubble 2.0 brewing out there.

It’s not just sleazy advertising, the money IBM has taken from cancer research for snake oil is downright fraud in my book

Advertising != Peer reviewed publication. Drinking Coke doesn't make you sexy and desirable, as suggested by their ads, just makes you gassy.

IBM is allowed some creative liberties in their mass-media advertising campaigns.

Question: how can the author know what Watson really is doing unless the author worked on Watson?

If you're going to try and explain the above to me, don't do it by explaining what Watson "really" is doing (unless you worked on Watson yourself). Explain exactly how you can tell the difference from the outside.

As far as I can tell all the author did was point out that Watson made a mistake (saying that Dylan's songs is about love fading) and this is not enough, since humans make mistakes too.

I am not sure if the author did this (he certainly doesn't mention it), but a lot of what constituted Watson at the time of Jeopardy was published as a set of 17 papers called "This is Watson" in the IBM Journal Of Research and Development.

Other than that too, I think you can make a reasonable assessment of any NLP system at this level of abstraction by looking at what is cutting edge in NLP today (by following conference proceedings etc).

It's not new! IBM does what it's best at which is advertising.

"Recently they ran an ad featuring Bob Dylan which made laugh, or would have, if had made not me so angry." Wait, is Roger Schank an AI trying to convince humans that AIs are impotent and harmless? Pretty sneaky, Schank-bot.

I worked on data systems that fed weather data into Watson.

IBM's technology, and IBM's marketing are very different beasts. The marketing is somewhat trivial, but the reality of ML and the Watson data systems is incredible. They are pumping a huge amount of data into it and they have data scientists doing incredible things with the data already. It is the largest growing segment of the company (and maybe the only growing segment of the company.)

The marketing of AI, and the realities of ML are always going to be disconnected. Sure, you will likely just get an email that says routine maintenance has been ordered on your elevator, not a box in the corner that tells you in a sexy, clear, non-robotic voice. As for Bob Dylan's songs... Christ. AI winter isn't coming unless the term AI gets squashed and people start calling it ML. This article is pretty FUD.

The problem is that all of this AI and ML is being used to target Ads better. You don't need AI and ML to do this, it is a solution looking for a problem.

Your example about routine elevator maintenance does't need AI at all. Whoever logs the elevator maintenance can just press a button to send a notification message. Hell, it can be wired up such that when you submit a notification notice, it automatically sends an email. They probably have condo management software that does this already.

The idea of the elevator maintenance from the IBM commercial is that it can use ML to detect that there will be downtime that could be solved by preemptively calling for some maintenance. I guess "routine" was a bad word choice.

Speaking for OP - I think the idea was that ML would determine that your elevator was in need of maintenance and place the order without human intervention.

Although the word "routine" belies that, but so.

Exactly. We already know when the elevator needs to be maintained. If it isn't being maintained, it's not because people don't know, it's because they are intentionally NOT doing the maintenance, maybe because it costs $120/hr minimum 6 hours.

Reply on news.ycombinator.com