IBM Has a Watson Dilemma | Hacker News Reader

IBM Has a Watson Dilemma

wsj.com

43 pointsdenzil_correa45 comments

Like much of IBM, Watson was/is a good platform for doing certain tasks, but blown completely out of proportion by IBM marketing. Not to mention that they now explicitly use it as a branding tool ("Watson Cloud", "Watson Health" etc.)

On the one hand, you have Google open sourcing AI tools, courses, adding TPU-enabled machines on their cloud and using their own AI tools to do some pretty neat stuff. On the other, you have IBM marketing deliberately deceiving ordinary consumers as to what its capable of. Its not that hard to take a side.

Disclaimer: I do not currently work for either company

I don't understand your point here.

Google is not a competitor to IBM and so there is no "choice" to be made between the two. Sure Google has open sourced some great technologies (as have many others) but they are raw, low-level tools for Data Scientists to use. They aren't OOTB products ordinary people can use.

I thought one of Watson's main projected uses was healthcare. Isn't Google's stake in healthcare via DeepMind actually make them competitors in that sense? Unless I'm missing something here, I have not researched Watson a lot.

They're competitors in the sense that neither are worth anything to the average medical Doctor at this point.

I don't mean this as a disparagement, merely as a dispassionate observation. Doctors need alerts about contra-indications of drugs they and their patients' other doctors have prescribed more than they need AI-derived diagnoses, for example.

I regret having only one upvote to give you. The state of information flow in today's clinical settings is depressingly bad, especially when you see what computers can do today. The focus on trendy techs we are experiencing in hospitals really reflects how backwards people have their priorities.

Google absolutely competes with IBM in the cloud hosting space and MLaaS space with offerings like ML Engine.

I have a friend who is an OB/GYN Oncologist in Indianapolis. Her hospital had IBM Watson Health on campus last year, and she mentioned to me in passing last month that they finally had kicked IBM off of their (learning) university's campus. When I asked her why, she said:

"Often times, Watson would recommend courses of treatment that would be completely incorrect, if not detrimental, to the patient. It became more of a hassle than a learning tool, and since the Residents are operating under my license, it's 'my way or the highway.'"

Hell, I went to an urgent care over the weekend, and they were still using a old IBM e-series application built with COBOL -- and when the doctor was inputting the drugs to prescribe, it warned him about potential interactions between the different drugs -- not something I had seen the EPIC system that my normal doctor uses do.

I still feel that there's a lot of good growth to be had for data scientists in the Health Care field...but, Watson itself is probably not the answer, and IBM should probably get rid of Ginni Rometty and find a new CEO, unless they want to continue cutting IBM down to a skeleton of what it used to be.

I worked at Epic years ago, but even then it was able to warn about drug interactions. But those warnings were configured on a site-to-site basis; your normal doctor's hospital (or practice) may not have configured drug interaction checks.

Not only that, but if the software at the Urgent Care is that old, it may be raising false flags on things that are no longer considered clinically relevant and just present a hassle for the provider to click through.

The age of the software would not reflect the quality of the data. I'm positive that these alerts are updated with new drugs and contradiction logic continuosly.

You are right in that the age wouldn't necessarily correlate with quality, but I can speak from first hand knowledge about the inertia that different EMRs bring. A lot of these drug interactions are mild precautions or the evidence changes with time. Ideally what is supposed to happen is a multidisciplinary committee reviews the latest evidence and continuously updates the interactions. What often happens though is that there isn't enough 'oomph' to override a previously established notification so inertia wins. It's often not until a major EMR upgrade where things get more closely looked at and some of the less important flags/notifications get weeded out out. Definitely true that new major interactions get added all the time and usually in a timely fashion. As you can also imagine there is a medico-legal aspect of removing a previous flag, so often it's easier to just leave it and make the provider do an extra unneeded click.

IBM's problems predate Rometty becoming CEO.

Glad to see the broader market finally catching on to the marketing fluff train that was “Watson”.

On the one hand IBM did an impressive job tapping into people’s desire for “big data stuff” as a lot of uninformed folks talked a lot about “hey we should get that Watson thing.”

On the the other hand messing with peoples lives via a bogus cancer curing machine is some messed up stuff. These Watson boondoggle projects diverted a lot of good resources that could have gone towards legit value-add research.

The whole Jeopardy thing was really cool and amazing marketing, but as anyone who tries “Watson” knows the “Watson” IBM has been trying to sell is not the Jeopardy thing.

What a horrible review. The cancer research isn't at all what you make it out to be. It's legitimate and does have some real proven value despite the recent backlash in certain parts of the press. It's not perfect but the advancements that have been made are significant.

It's easy to stand on the sidelines and throw rocks at other peoples efforts based on very little first hand information but please, this isn't twitter.

Not on the sidelines, I’m on the field. That’s why I’m glad to see the reality finally getting attention. As the article says, a lot of healthcare professionals that once were onboard have distanced themselves from IBM and these projects.

So is there a better oncologist’s assistant AI out there? Or is the sweet spot somewhere between picking music you’ll like, suggesting when you should leave for an appointment to manage traffic and playing chess or go?

I understand the dislike of the marketing, but something like Watson in medicine is a moonshot attempt. Can CNNs and whatever other stuff Watson has available to it simply not handle cancer? Or is it broken? Or is the literature and training data bad or there just isn’t enough of it? I’m really fascinated by the stories of it giving detrimental treatment suggestions, I could see it happening a couple times and word getting out (like a new file system corrupting data) but if that’s really normal then something is broken, right? Maybe it’s misunderstanding the training literature, which should be a correctable software bug. I’m just guessing.

There’s nothing inherently wrong with the technology.

However, doing ML well isn’t really about the ML as much as it’s about getting everything leading up to the ML right. That’s the difference between value-add applications of ML and an expensive mess. IBM didn’t understand that and hence the mess they’re in now with Watson.

"IBM didn’t understand that"

Can you explain a little further?

My experience with Watson is a manager asking if I had any project ideas for it because the company had “Watson time” that no one could think of a way to use. I tried to picture the chain of events necessary for the exchange to have taken place that wasn’t comical. I failed.

After being forced into ClearCase use at another company, I couldn’t help but imagine how much damage IBM sales has done to the software industry. How many inferior products have been propped up and resources drained by such dishonest sales practices and poor leadership?

I’ve been in similar positions, but you can’t blame IBM — the situation that you found yourself in has everything to do with decision making at your employer.

You could tell the same story or worse about Oracle, HP, etc. The difference is that IBM’s strategy of tying hardware to solutions has kept it viable far longer than its competitors.

I’d argue that Microsoft is worse — at least IBM treats your boss’ boss to steak and golf. Microsoft burrows in with bundled ELAs. The annual cost to carry support + maintenance on some products is going to be as high as 39% of product price!

> but you can’t blame IBM — the situation that you found yourself in has everything to do with decision making at your employer

I partially agree. I would say that both IBM and companies that get suckered by them are dysfunctional.

Watson is just IBM's way to market solutions based on ML. The whole Jeopardy game where Watson beat the human contestants was a big advertisement for the set of solutions. When I was in school a couple of professors went to IBM's office . They were expecting a technical breakdown of how the Watson system worked. Instead they just got some marketing that you would not need to have to create code to solve problems. The professors promptly left after that.

Watson as vaporware articles come up here about once a month. Not that the articles are wrong. Just a bit amusing.

So given that IBM’s only big commercial bet is (was?) Watson and it is clearly not going to drive significant revenue.. what do IBM do next?

They ballsed up cloud, they let the rather impressive DB2 wither, and their software suite is... meh.

Broadcom might be interested in acquiring them.

Broadcom ($90b market cap) is worth 70% of what IBM ($133b market cap) is worth..

If a company stops having good revenue sources their stock tends to drop..

They can always fall back on consulting; there will always be corps willing to pay them in order to learn how to do things the "IBM way"

http://archive.is/yoK65

Do we have a rehash count on this? Getting old...

Watson isn't what investors and the general public thinks it is. It's now more or less some classifier/machine learning as a service plus a dev integration team you can hire to pull it into your codebase (which by the way, is lots of $$$) It is not difficult for anyone on HN to spend a weekend, read a machine learning book, and replicate most of Watson's functionality. It's IBM marketing at it's finest.

I don’t know if what you are saying is true.

I do know that in the past, every time I try to corroborate a similar statement made in a similar tone I end up realizing the commenter was presenting a very narrow and biased, inaccurate, and occasionally slanderous view.

So I don’t invest that time and energy anymore. Life is finite.

> It is not difficult for anyone on HN to spend a weekend, read a machine learning book, and replicate most of Watson's functionality

Yea.... I'm sure that's exactly how it is.

Reminds me of https://news.ycombinator.com/item?id=678501

reminds me of https://news.ycombinator.com/item?id=537246

It's a "china" grade replica based on Django, from the commit message it's done alone by some not talented dude, cost far less than 6 months.

Watson is a term that is now so overloaded with hype that I tend to ignore it. It's worse than AI.

you are full of it. Are you sure you can replicate beating world class players on jeopardy in a weekend?

But that's not Watson is and I think this is his point. Sure they had a great model and data to solve that problem. But this doesn't scale automatically for other use cases. You'd have to create a model for your use case and find training data. Tensorflow might offer better tools for that for free than the ones being shipped with Watson.

While parent post is full-of-it on the general question of replicating Watson's full bag-of-techniques against client problems in a weekend, I believe the Watson Jeopardy victory was far less impressive than it seemed.

It's a very-constrained domain, and the challenge Watson won has never been opened to scrappy teams other than IBM. I wrote up my doubts way back in 2011:

https://memesteading.com/2011/02/16/ibm-watson-overprovision...

Here in 2018, I suspect a clever focused team could make a MacBook, with current dumps of Wikipedia/Wikidata and a few other key reference materials, competitive in most Jeopardy matches in a week or two. And, with 6 months to a year, as dominant as "Watson" was against the Jeopardy champs.

I've had the pleasure of interacting with the codebase that played Jeopardy! and from where I stand, it's still one of the most impressive and beautifully-orchestrated systems I've ever seen.

You are correct in your write-up though. Wikipedia titles are super helpful, where they exist.

> And, with 6 months to a year, as dominant as "Watson" was against the Jeopardy champs.

Was Watson dominant because it had a buzzer advantage, or because it could answer the questions better? I recall the other two guys furiously trying to hit the button on different occasions and sighing when Watson beat them out yet again. Thing is we don't know that Watson was better at answering the questions, only that it was better in the Jeopardy game play the way things were setup.

Please don't cross into personal attack regardless of how wrong you think another comment is. That only makes this place worse.

https://news.ycombinator.com/newsguidelines.html

You should do that then. After you’re done you can staff up 400 sales people at $120000 salaries to snatch one of IBM’s clients to a $60m contract and make a cool $10m+ profit.

Worked in a similiar setup as client of IBM and can confirm what you mentioned, where they tend sell consultancy on some ‘enterprise’ functionality that is already taken over by open source or better services like offered by Azure or AWS

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."

https://news.ycombinator.com/newsguidelines.html

The i-could-do-this-in-a-weekend trope is the mother of shallow dismissals. Please don't post like that here.

Reply on news.ycombinator.com