Comment by saosebastiao | Hacker News Reader

Comment by saosebastiao | Hacker News Reader

I've been baffled by this as well. I can understand why deep learning has done well in fields that can roughly be described as sensory perception, but it has never improved on basic random forests or SVMs for problems in my domain. And we have lots of data (at least more than can fit in an R instance).

Even taking data size out of the picture, functionally it is not there yet for most tasks. Maybe it will be in the future, but the big problem with it is that with n neurons, you have n^n possible topologies, and finding the right neural topology is a major optimization problem that we're only barely learning basic human heuristics for.

I'm willing to bet the deep learning thing is just one more Neat fad that will eventually cause disillusionment at its lack of results, reverting us back to the Scruffy view that intelligence is far too complex to be described holistically by small sets of simple algorithms. The great thing about the Scruffy philosophy is that it isn't derogatory...deep learning will always have a place as a tool in its tool set. It merely doesn't hold unreasonable expectations.

>I'm willing to bet the deep learning thing is just one more Neat fad that will eventually cause disillusionment at its lack of results, reverting us back to the Scruffy view that intelligence is far too complex to be described holistically by small sets of simple algorithms. The great thing about the Scruffy philosophy is that it isn't derogatory...deep learning will always have a place as a tool in its tool set. It merely doesn't hold unreasonable expectations.

Deep learning is already a Scruffy fad. It basically says, "Hey, let's use a really huge hypothesis space of circuits that often includes a heavy prior towards convolutions." Gradient descent is a Neat principle, but the whole point of things like improved training methods, new objective functions, and convolutions was to deal with the exploding-gradient problem.

Deep learning didn't come up with its own Neat principle, it invented Scruffy methods to apply a Neat principle to a really fucking huge hypothesis space, so long as you've got a pretty big dataset.

What is your domain?

Transportation, Logistics, Supply Chain Management, (physical) Operations.

I suspect the reason why deep learning has done so poorly in my domain is that the underlying data is a result of things that are very poorly abstracted as a "function". We have lots of discrete events, stateful buffering, hard non-linearities, discontinuities, numerical bounds, etc. It's more like learning business rules and physical process design than learning a mathematical function. This is part of the reason I don't see deep learning being a holistic solution for self driving cars...once you get past sensory perception and simple 2d path planning, driving is more of a rule based process than anything.

That being said, ML tends to be a pretty niche technique for us anyway. If a process and its components are well known and understood, we tend towards solutions that come from Operations Research over Machine Learning. It is only when things are poorly understood that we use ML (example: predicting product demand fluctuations based on media coverage or predicting truck arrival times given severe weather patterns and traffic backups). PGMs do really well here, but are far more difficult to understand, formulate, and train...for most tasks Random Forests are almost always Good Enough(TM).

+1 For real world business problems that I most frequently encounter doing consulting, it's hard to beat Random Forests and/or Gradient Boosting. Truth be told, most business problems I encounter turn out to be largely helped by good old linear models.

Agreed! I have often done more sophisticated analysis and then stepped back and concluded that a simpler analysis was actually better for the business. It moved the business into a better place for informed decisions and gave them simple (analysis backed) rules of thumbs that every manager/director/ VP could understand and use by just checking a couple numbers and doing a simple easily remembered bit of math.

Understandable models with clear intervention points are what most businesses seem to need once you get to digging around in their operations, customer and sales data.

Is there any example of PGM applying to this field?

Reply on news.ycombinator.com