Hacker News Reader

Comment by nullptr_deref | Hacker News Reader

OpenAI staff in product and applied research. Didn't see any from the hardcore agi/safety research. So there is some internal division already.

View on HN

Who is actually responsible for the real innovation there? I read Ilya was a co-author on the papers that made GPT and Dall-E possible in the first place. Is anybody in that weight class publicly siding with Sam, or is it all people working on APIs etc.?

Going through GPT-4 contributors list Ilya and Sam are both under additional contributors with nothing specific attached to their name. I take this to mean they didn’t contribute anything specific but OpenAI thanks them for their service. Greg brockmann is infra lead, Jacob Pachoki is overall lead of GPT4! and optimization lead, Szymon Sidor is optimization vice lead and these are all people who publicly resigned with Sam Altmans ouster.

Now going through ChatGPT contributors list, there is no mention of Ilya or Sam, but Greg, Jacob and Szymon are mentioned.

When it comes to the GPT3 paper: Ilya is thanked as an advisor, Greg sam or the rest are not mentioned. It seems like most of the leads created Anthropic.

In my own observations, Ilya stopped doing much for OpenAI since around 2019, he was mostly an advisor and gave talks here and there. He was most likely not responsible for the innovation behind OpenAI. An early codebase of GPT-3 was credited to Prafulla Dhariwal, and its lead was Tom Brown. ChatGPT seems to have been led by John Schulman. GPT-4 by Jacob Pachoki. Greg seems to be involved in setting up the infrastructure code for large GPU clusters and training. Clearly the folk with the skills to build and innovate in OpenAI prefer Sam as CEO and that says a lot about Sams leadership

I get that he didn’t work on these things day to day, but ultimately who was responsible for the innovations that made these programs possible in the first place? For example, was anyone at OpenAI involved in the creation of transformers?

Even Ilya was not involved in the creation of transformers…

Honestly by your logic, NVIDIA are the real innovators, without their hardware none of this would be possible (including the very first Deep Learning Win, known as AlexNet)

I'm not proposing any logic. I'm just trying to figure out who if anyone at open AI actually pioneered the development of this technology, rather than simply worked toward its implementation at scale.

The reason it matters is because if they are just building all this stuff based on ideas taken from other outside researchers' papers and throwing money toward making it happen, OpenAI's moat is smaller, and it's easier for a bright group of competitors with lots of funding to catch up.

OpenAI has not invented much of any new ideas. None of them worked on transformers, or diffusion or a host of other technologies that power their models. (though one of the leads at OpenAI built PPO while in his PhD). But they’ve perfected the art of training and deploying models, which I think is their moat. It still shocks me that their speech recognition model is miles ahead of Siri (that struggles heavily with my accent) and even Alexa. How is it that 2 companies that have invested billions of dollars into these products, end up with worse results than a small team in startup whose main focus isn’t speech recognition?

Almost like how Toyota didn’t invent the car, but perfected the art of building reliable cars, I see OpenAI as perfecting the art of the building ML models that just work in the real world

OK thanks, that makes sense. Who at Open AI would you say was most responsible for perfecting that training?

I’m not sure, it’s hard to say unless you’re in OpenAI. Also it’s almost likely a team, not 1 person

I wouldn't interpret missing interactions on twitter as more than people not using twitter.

Twitter peaked at 368 million users 2022. It is among various analysts expected to drop 4% of users to reach approximately 2020 level in 2023, and another 5% more users in 2024 to reach 2019 level [0]

That's not enough of a loss to motivate discarding twitter conversations.

[0] https://www.bankmycell.com/blog/how-many-users-does-twitter-...

Reply on news.ycombinator.com