Hacker News Reader

Comment by danShumway | Hacker News Reader

Agreed. Throwing more computing power at the problem is a cheap way to make it look like we're making progress; our AI should be efficient enough to run on limited hardware. Not to say that ML/DL isn't useful, but I think that whatever the next revolution is in AI is likely to come from a completely different direction.

I'm not a researcher either, so it's difficult for me to articulate exactly what problems I see, but (shameless self plug) I did write an article a little while ago attempting to: https://medium.com/@danShumway/modern-ai-techniques-arent-wo...

View on HN

It's worth noting that the training-time inference-time distinction matters here. Most of the fancy tech you've heard of recently (Semantic Seg, Pose, Localisation etc) can be pretty easily optimised for fast inference, and indeed it's not really so much of a research focus because it's so tractable (see MobileNets, v2, etc). Training, however, is still quite daunting.

That's a good point.

I tend to be fairly dismissive of inference because what you end up with is a highly specialized algorithm rather than something that can easily continue to adapt. I suspect inference would probably fall into Jordan's category of "things that we call AI but probably shouldn't."

But that's not to dismiss how important fast/cheap inference has been in allowing companies to actually build things with AI.

Also even on the learning side, perhaps the raw task of making things faster isn't really that hard

https://eng.uber.com/accelerated-neuroevolution/

What's special about 2018 computing power? Neural networks have been declared dead several times in the past. Because they couldn't do anything interesting with the measly computing power available at the time. Currently the biggest RNNs have a few thousand neurons. Which is absolutely puny compared to even animal brains.

Of course this is defending a strawman. I don't know anyone that said it was just a computing power issue. In fact I think most people vastly underestimate the role of computing power. Even algorithmic improvements are enabled by computers letting researchers do experiments that would have been impossible before. And by doing lots of experiments they gain intuition about the problem, that they wouldn't develop in a vacuum.

The real fallacy is calling neural networks the analogue of animal brains. We still have a lot of research to accomplish in this sphere.

They aren't. You're looking for "hebbian learning". Very different from neural networks.

In 50 years people will laugh about how poorly these things were named. Just like we now laugh about what symbol we chose for a source of electrons in a circuit : "+". Whoops.

You both seem to have really confident opinions given that you admit you have no expertise in the field.

Reply on news.ycombinator.com