Hacker News Reader

Comment by ve55 | Hacker News Reader

Something can be both legitimately revolutionary/interesting, but also significantly over-hyped and misrepresented, often with strong for-profit incentives. Some recent good examples of this include progress in cryptocurrencies, decentralization, and ML/AI.

View on HN

Sure, but many people disagree that ML itself is revolutionary. The basics of it were known (referred to, quite appropriately, as 'data mining') as early as the 1990s and perhaps earlier. We've added a smattering of new techniques since then, and compute power has been expanded via GPGPU, but there was no "revolutionary" shift in the field. Even multi-layer ("deep") neural networks are very old tech.

As evidenced by gpt3 we still don’t know the best way to use deep learning. More data improved the outcome dramatically. What else might surprise us?

The smattering of new techniques seem to have made the difference between success on toy problems vs. being able to match or exceed human performance on many difficult tasks. So while naysayers are correct that "the math hasn't changed since the 90s!", enough has changed to make calling DL a paradigm shift accurate.

For reference, I can now get an intern to images for a few hours, then train a black box algorithm to automate their efforts in another few hours. This algorithm is sensitive, brittle, and may have perfomance issues, but it's still already orders of magnitudes better than what took days or months of effort prior. That to me is a revolution, regardless of the math.

> then train a black box algorithm

This is part of the problem. Finding a the value of a few hyperparameters is hardly something I consider interesting science.

In the recent success stories on images, audio, and text, it's not "a few hyperparameters" by any stretch of the imagination.

That's like saying "finding the sequence of assembly instructions / nucleotides / ... is hardly something I consider interesting science"

Come on, really?

Electric motors and lithium batteries aren't new either. So much for the EV revolution, nothing to see here.

I assume you're being sarcastic, but there actually isn't anything to see. Plug-in hybrids blow any EV out of the water and will do so the foreseeable future. They're cheaper, lighter, just as efficient on short trips, and much more practical on long trips.

Hybrid vehicles are the practical option today. Pure gasoline vehicles are outmoded and EVs are all hype.

And I'll assume you've never driven an EV, because almost everyone who has purchased an EV will never go back to an ICE vehicle. An EV purchase is a ratchet. Hybrids make a lot of sense for some people today, but battery electric vehicles are the inevitable future.

Perceptrons are indeed old tech. But try training models for even something as simple as handwriting recognition using techniques from the 90s and modern techniques but with the same training set and compute resources. You'll get much better results with the modern stuff.

Reply on news.ycombinator.com