Comment by andy99 | Hacker News Reader

Comment by andy99 | Hacker News Reader

Parent Full thread

I'm curious to know the answer to this question as well. I have done a fair bit of work working with organizations to identify ML use cases. When we looked at it from a business process perspective, honestly it didn't go very well. Trying to find company process specific interventions, especially in the format of building a funnel to priortize which to move forward, rarely surfaces unique or game changing ideas. We usually ended up generating a list of things where either ML played a minimal role, something more simple would have been better, or you'd need AGI.

What I've seen work better is a product approach, where ML is incorporated as a feature (rarely but possible the centerpiece) of a full solution for an industry that provides a new way of doing something and the value that comes with it. The caveat is that this is hard and takes up front R&D and product market fit research that any product would. It doesn't happen in a series of workshops with representatives from the business.

This Dropbox story is an obvious counterexample, and really looks like the mythical "low hanging fruit" that we always want to identify in ideation workshops. But I'd be careful trying to generalize a process for identifying ML use cases from it.

I’ve worked on ML across several large e-commerce firms, and two patterns I have seen along the lines of your comment:

1. many organizations dismiss ML solutions without actually trying them. Rather, if one hack week style prototype doesn’t work on the first try, it’s chalked up to “over hyped AI” and never sees the light of day. Organizations that succeed with ML don’t do it that way. Instead they ensure the success criteria are stated up front and measured throughout, so you can see why it didn’t work and iterate for v2 and v3. “We spent a bunch of R&D expense on magic bullet v0 and it didn’t succeed immediately” is a leadership and culture problem - you probably can’t succeed with ML until you fix that.

2. Many companies have no idea how to staff and support ML teams, and go through various cycles of either taking statistical researchers and bogging them down with devops or taking pure backend engineers and letting them do unprofessional hackery with no clarifying product quality ML expert in the loop.

You need a foundation of ML operations / infra support that cleanly separates the responsibilities of devops away from the responsibilities of model research, and you must invest in clear data platform tools that facilitate getting data to these teams.

If an org just figures they can throw an ML team sink or swim into an existing devops environment or they can require an ML team to sort out their own data access, it’s setting ML up for disaster - and again you’ll get a lot of cynics rushing to say it’s failing because ML is just hype, when actually it’s failing due to poor leadership, poor team structure and poor resourcing.

Yeah, for one thing, the scale of Dropbox may make this a uniquely worthwhile investment for them. Many apps have similar kinds of speculative caching features that could be optimized with predictive modeling, but the same cost-benefit analysis might show that it saves $1.7k/year instead of $1.7M, less than the cost of the feature's development and maintenance.

@andy99, @setib: we're a boutique that helps large organization in different sectors and industries with machine learning. Energy, banking, telcos, retail, transportation, etc. These organizations have different maturity levels and their functions expect different deliverables.

The organizations range on the maturity level from "We want to use AI, can you help us?" to "We have an internal machine learning and data science team that's overbooked, can you help?" to "We have an internal team, but you worked on [domain] for one of your project and we'd like your expertise".

For the expectations, you can deal with an extremely technical team that tells you: I want something that spits JSON. I'll send your service this payload and I expect this payload. So that's a tiny part.

Sometimes, you have to build everything: data acquisition, develop and train models, make a web application for their domain experts with all the bells and whistles, admin, roles, data management, etc. I wrote about some of the problems we hit here[0].

The point is that, finding these problems is an effort that requires a certain skill/process and goodwill from the clients. We worked on a variety of problems.

- [0]: https://news.ycombinator.com/item?id=25871632

Reply on news.ycombinator.com