Hacker News Reader

Comment by greatpostman | Hacker News Reader

Funny that a relatively small highly competent company like openai has created a stir that none of the big companies could. The 10x engineer theory is alive and well. A tale as old as time, large “prestigious” corporations incapable of innovating, probably due to management being unwilling to cannabalize their own jobs and knowledge cough google cough

View on HN

The odds were truly stacked against this scrappy group of well-connected billionaires, supported by the largest tech companies in Silicon Valley, able to hire the top talent in the world, originally touted as a non-profit that would democratize AI instead of gate-keeping it for the profit/benefit of the aforementioned.

At least they didn’t say “sike” as soon as the tech was good enough to monetize and pivot to being a for-profit tech company in itsel- oh.

Agree they had an unlimited budget and hired true top talent. But my point still stands, none of the big companies are spinning up a group of 200 people in a new org and releasing game changing products. They can only do it by funding external groups. Facebooks whole business model was hiring expensive engineers that are good at executing and just copying any new competitors product.

Google has their own gpt AI, it’s just not released to the public.

Exactly, management is still riding the gravy train and won’t risk their now obsolete business model

You do realize Google relaeses more products than anyone, right? (In fact, too many.) But with just an AI chatbot, there really isn't much of a product in it's current state. But lets say there is this massive product that Google is hiding, I'm really struggling to understand how Google releasing it risks their "now obsolete business model." I mean, what exactly is your claim here? Google has this killer product, but they aren't releasing it and putting ads on it because... they want to let someone else do it? But they _are_ still publishing the very research that will obsolete thier business model at no charge?

https://www.theverge.com/2022/12/14/23508756/google-vs-chatg...

So what's your point exactly? First OpenAI has 10x engineers that Google doesn't, now Google has the 10x engineers, as you acknowledged you know OpenAI used Google Brain's trsnsformer architecture, but it just isn't productising fast enough for you? You do realize OpenAI hasn't made a dime yet, right? So your so called 10x dream is just time to market? Not inovation, or releasing a real product?

I think this is true across nearly every industry. It’s easier and safer to invest in another business than to try and do it in-house, so they do. Maybe interest rates might change that approach in the future.

Amazing points. The day of the dorm-room student billionaire is already over. This said, GP's comment has some value. If we take "relatively small company" to mean number of employees (although this wasn't made clear) then there is some truth to the idea that a new, unencumbered company can do more than an established one. But I feel GP is minimizing Microsoft, Google and Nvidia's contributions to ML research, and perhaps overplaying OpenAI's role in pitting it against other players when clearly everyone is building on each other's result. GPT uses a transformer architecture, which was created by Google Brain, and the first GPT paper alone cites 71 other papers, each, in turn, citing their own...

It's less about engineering ability. For sure, OpenAI has phenomenal technical talent, but large tech co's do too, and in fact, many of them have internal developments that match (maybe exceed) GPT3 / ChatGPT.

It's more about organizational bureaucracy, innovator's dilemma, organizational priority, risk tolerance given the scrutiny they're under, etc.

> It's more about organizational bureaucracy, innovator's dilemma, organizational priority, risk tolerance given the scrutiny they're under, etc.

The paypal argument

> internal developments that match (maybe exceed) GPT3

Had a funny google prompt today which echoed wrong results by their ML model: https://www.google.com/search?q=single%20neighbor%20country

> It's more about organizational bureaucracy, innovator's dilemma, organizational priority, risk tolerance given the scrutiny they're under, etc.

Agree. Being on the right side of all those items also attracts more top talent.

The “Innovator’s Dilemma” mostly is about “low end disruption” when a product by the leader in a category “over serves the market”. Nothing about this fits that pattern.

You do realize that Google invented the transformer architecture, and OpenAI has simply scaled it up?

Yeah I am completely aware. They invented it and search got even more useless.

I fail to see how that is an argument against what the parent comment said. Google invented transformers and OpenAI made it into something compelling. More compelling than what Google has done with it.

Google invented the engine and OpenAI took that engine and built a car. Both deserve credit.

I agree with that, but ultimately how the transformer architecture is used in products is what's going to change things and create new value for humanity.

The difference is that OpenAI is well, open and usable and Google’s offerings are not.

It seems to me that such a split could be because OpenAI has decided that ChatGPT needs to be public to get hype, partners, investment, training data, etc. Google many not need or want those things.

It's definitely possible that ChatGPT is way better than LaMDA, but I wouldn't take the availability of a public demo as proof of that.

Google apparently has a bot that can put ChatGPT to shame, it’s just not released to the public.

Sure they do.

https://www.theverge.com/2022/12/14/23508756/google-vs-chatg...

Having something and failing to act on it is even worse than not having the thing. It means you're paralyzed and calls into question your ability to act on anything at all.

Is that the one where one of their employees felt it was sentient?

I have trouble understanding why OpenAI contributions are always so much undervalued, I get it that they didn't hold their promise of being "Open", still we owe them most of the recent breakthroughs is AI

PPO, RLHF, Diffusion models, CLIP (which has been the root of all text2img techniques), GPT were all either created or driven by OpenAI.

Google, Meta, Microsoft all had enough resources to produce the same results but most of the time they only replicated OpenAI's work afterward

OpenAI's ability to deliver when Google couldn't has been spectacular and puts Google to shame.

Sama is an Entrepreneur. Most major stable companies are typically run by the MBA types, Execs etc who won't be taking any major risks. Risking their reputation, rocking the boat aka Status Quo etc. If the risk doesn't pay off they are screwed. If they maintain the status quo it will take a long time before consequences catch up.

That's why Elon, Zuck, Steve Jobs etc make major singular decisions without much bureaucracy standing in their way.

> The 10x engineer theory is alive and well.

> due to management being unwilling to cannabalize their own jobs

Sounds like a 0.5x engineer could still have produced the desired output if the alternative is a manager that resisted change. Was it incredible engineers, or bad managers? Both?

>probably due to management being unwilling to cannabalize their own jobs and knowledge

Disagree.

The reason Google does not innovate anymore wether AI or elsewhere:

https://www.youtube.com/shorts/0UXIFoCbXog

They have less reputation to risk. Big tech and so on can't put out AI prototypes as easily without ending up on the front page of NYT when it comes out with some egregious learned biases around race, sexism, Nazis, and so on.

I find this very hard to believe. Microsoft is also working on integrating chatgpt. But chatgpt itself probably could never have been built within Microsoft. Managers gate keeping their own jobs won’t allow it

How quickly we forget Tay.

https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

But the big companies have infinitely more data, which will be more useful in the long run

Actually it turned out to be the opposite, vanilla data scraped from the internet is feeding these models

How is that the opposite? Big tech companies have all the vanilla data scraped from the internet, plus WAY more. Obviously both together will be more valuable than just the former.

A team of 10 talented engineers can scrape and store that data. It won’t be perfect, but it’s nowhere near the barrier that people assumed.

1 engineer can handle that alone. The Pile is under 1TB. The tech is there to do it from a phone in under a month.

> A team of 10 talented engineers can scrape and store that data.

How do you scrape and store the nonpublic data held by any major corp?

None of the current crop of big companies are being led by founder-CEOs - except for Meta. They're all being led by manager-CEOs.

Manager-CEOs, historically, don't have the stakes, the vision, or the sheer pull to cannibalize $100B existing business lines with a brand new, untested product.

Big Tech's bigness will be its biggest liability. A chatGPT-like tool, for instance, competes against Google's search products. And Google can't monetize it with 10+ ads on every page.

Google et al will dawdle and hem and haw and watch nimbler competitors take away market share. Tale as old as time.

Google’s CEO is unique within BigTech of lacking any vision or any real accomplishments. I still wonder how he ever became a CEO.

No one claims that Nadella, Zuckerberg, Cook, or Jassy lack vision.

Not a knock on Nadella, Cook or Jassy, but do they have the pull or power to go through multiple quarters of declining revenue while they pull off the AI strategy?

Zuck is trying that with his metaverse and getting so much flack from everyone. But he can still pull that off as founder-CEO and the board structure.

ChatGPT is not a threat to any of their business models. It is a threat to Google.

Can you imagine how much better Siri would be if ChatGPT were integrated into it?

Apple and Microsoft have both been able to remain relevant since 1980. They both know how to pivot when necessary.

Reply on news.ycombinator.com