Hacker News Reader

Comment by Tycho | Hacker News Reader

Recently heard a resident machine learning expert describe GPT-3 as 'not revolutionary in the slightest' or something like that.

View on HN

It's not, but it showed that we can get a magnitude better results by adding a magnitude more data.

To be honest, it's not where I'd like to see efforts in the field go.

Not because I'm afraid of AI taking over, but because I'd rather have humans recreate something comparable to a human brain (functionality wise).

Who knows, maybe in a few years you will be amazed at the new universal transformer chip that runs on 20W of power and can do almost any task. No need for retraining, just speak to it, show it what you need. Even prompt engineering has been automated (https://arxiv.org/abs/2101.00121) so no more mystery. So much for the new hot job of GPT prompt engineer that would replace software dev.

I was skeptical before, but now i open to this idea

we've been making these since the beginning of time, we call them humans

Human's want health insurance and 40 hours work-weeks. The super-smart AGI that will exist 20 years from now won't.

It's revolutionary in costs, and delivers for every dollar spent.

I think that they're correct saying that GPT-3 isn't revolutionary, since it just demonstrates the power of scaling. However I would argue that the underlying architecture, the Transformer (GP(T)), is/was/will be revolutionary.

It's not revolutionary, just a typical-but-notable iterative step in NLP. Which is fine!

I wrote a blog post on that a few months ago after playing a bit with GPT-3, and it holds up. https://news.ycombinator.com/item?id=23891226

Reply on news.ycombinator.com