The industry structure of LLM makers
calpaterson.com
calpaterson.com
some ppl don't believe this cus it seems crazy.
anyways, yes they're trying to make their own chips to not be beholden to nvidia, and are investing in other chip startups. And at the same time, nvidia is thinking that if they can make an AI, why should they ever even sell their chips, and so they're working on that too.
This sounds like a perpetual motion machine or what we heard over and over in the 3d printing fad.
We have natural general intelligence in 8 billion people on earth and it hasn't solved all of these problems in this sort of instant way, I don't see how a synthetic one without rights, arms, legs, eyes, ability to move around, start companies, etc. changes that.
Unfortunately for AI in general, LLMs are forcing data moats, either passive or due to aggressive legal attack, or generating so much crud data that the good data will get drowned out.
In fact, I'm not sure why I continue to uh, contribute, my OBVIOUSLY BRILLIANT commentary on this site knowing it is fodder for AI training.
The internet has always been bad news for the "subject expert" and I think AI will start forcing people to create secret data or libraries.
So if you had that great idea that takes a full day to prototype, hence you never bothered, an LLM can whip out something reasonably usable under an hour. So, it will make idea-driven people more productive. The problem is, you don't become a high-level thinking without doing some monkey work first, and if we delegate it all to LLMs, where will the next generation of big thinkers come from?
Except that it is actually what humanity and these 8 billion people are doing, making each successive generation "better", for some definition of better that is constantly in flux based on what it believed at the current time.
It's not guaranteed though, it's possible to regress. Also, it's not humanity as a whole, but a bunch of subgroups that have slightly differing ideas of what better means at the edges, but that also share results for future candidate changes (whether explicitly through the international scientific community or implicitly through memes and propaganda at a national or group level).
It took a long time to hit on strategies that worked well, but we've found a a bunch over time, from centralized government (we used to be small tribes on plains in in caves) to the scientific method to capitalism (and whether it's what we'll consider the best choice in the future or not it's been invaluable for the last several centuries), they've all moved us forward, which is simple to see if you sample every 100 years or so going into the past.
The difference between what we've actually got in reality with the uman race and what's being promised with GAI is speed of iteration. If a areal GAI can indeed emulate what we have currently with the advancement of the human race but at a faster cycle, then it makes sense it would surpass us at some point, whether very quickly or eventually. That's a big if though, so who knows.
> then they can immediately use it to make itself better.
"AGI" is a notoriously ill-defined term. While a lot of people use the "immediately make itself better" framing, many expert definitions of AGI don't assume it will be able to iteratively self-improve at exponentially increasing speed. After all, even the "smartest" humans ever (on whatever dimensions you want to assess) haven't been able to sustain self-improving at even linear rates.
I agree with you that AGI may not even be possible or may not be possible for several decades. However, I think it's worth highlighting there are many scenarios where AI could become dramatically more capable than it currently is, including substantially exceeding the abilities of groups of top expert humans on literally hundreds of dimensions and across broad domains - yet still remain light years short of iteratively self-improving at exponential rates.
Yet I hear a lot of people discussing the first scenario and the second scenario as if they're neighbors on a linear difficulty scale (I'm not saying you necessarily believe that. I think you were just stating the common 'foom' scenario without necessarily endorsing it). Personally, I think the difficulty scaling between them may be akin to the difference between inter-planetary and inter-stellar travel. There's a strong chance that last huge leap may remain sci-fi.
Physical reality is the ultimate rate-limiter. You can train on all of humanity's past experiences, but you can't parallelize new discoveries the same way.
Think about why we still run physical experiments in science. Even with our most advanced simulation capabilities, we need to actually build the fusion reactor, test the drug molecule, or observe the distant galaxy. Each of these requires stepping into genuinely unknown territory where your training data ends.
The bottleneck isn't computational - it's experimental. No matter how powerful your AGI becomes, it still has to interact with reality sequentially. You can't parallelize reality itself. NASA can run millions of simulations of Mars missions, but ultimately needs to actually land rovers on Mars to make real discoveries.
This is why the "last company" thesis breaks down. Knowledge of the past can be centralized, but exploration of the future is inherently distributed and social. Even if you built the most powerful AGI system imaginable, it would still benefit from having millions of sensors, experiments, and interaction points running in parallel across the world.
It's the difference between having a really good map vs. actually exploring new territory. The map can be centralized and copied infinitely. But new exploration is bounded by physics and time.
The magnum opus of this notion is the Von Neumann probe.
With the entire galaxy and eventually universe to run these experiments the map will become as close to the territory as it can.
Current GPU manufacturing is probably one of the most complex human endeavors we've ever created. You need incredibly precise photolithography, ultra-pure materials, clean rooms, specialized equipment that itself requires other specialized equipment to make... It's this massive tree of interdependent technologies and processes.
This supply chain can only exist if it is economically viable, so it needs large demand to pay for the cost of development. Plus you need the accumulated knowledge and skills of millions of educated workers - engineers, scientists, technicians, operators - who themselves require schools, universities, research institutions. And those people need functioning societies with healthcare, food production, infrastructure...
Getting an AI to replicate autonomously would be like asking it to bootstrap modern economy from scratch.
The second we have humanoid robots that can do maintenance on themselves as well as operate their assembly lines and assembly lines in general will be a massive shift.
I think the baseline for that will be a humanoid robot that has the price tag of a luxury car and that can load/unload the dishwasher as well as load/unload the washing machine/dryer and fold and put away clothes. That will be total boomer-bait for people who want to 'age in place' and long term care homes in general.
Once we have that we can focus on self-replication on the micro-scale. There is tremendous prior art in the form of ribosomes and cells in general. A single cell hundreds of millions of years ago was able to completely reshape the entire face of the earth and create every single organism that has come and gone on the Earth. From fungi to great whales to giraffes, jellyfish, flying squirrels, and sequoia trees the incredible variety of proteins in a myriad of configurations that life has produced is remarkable.
If we can harness that sort of self replication to make power our economy it will make the idea of bootstrapping the economy on this world and others much easier.
This is why our team developing The Ac28R have taken a completely new approach. It's a new kind of AI which can write complex accurate code, handling everything from databases to complex financial models. The AI is based on visual specifications which allow you to specify exactly what you want, The Ac28R’s analytical engine builds all the code you need - No guesswork involved.
> Most LLM users seem willing to change from Chat-GPT to Claude, for example.
Talk to people who aren't engineers and it's all ChatGPT. Many don't even know about the concept of an LLM or a provider, just literally "ChatGPT". The South Park episode where they parody this stuff? They call it ChatGPT. The stuff students use every year to help with homework? ChatGPT. The website that "chat.com" redirects to? ChatGPT. And cai has cornered to market on horny/lonely male teens.
The moat here is the broader consciousness that a very very large population of people have adopted. Articles like this take something technical -- the cost of switching over to an LLM, which is cheap -- as an assumption that it will happen, without taking into account just how difficult it is to change social forces.
This doesn't mean ChatGPT will forever be what people use. Maybe it will fail spectacularly in a year. But it's OpenAI's game to lose here, not the other way around.
For airlines availability with a preferred carrier is not guaranteed, and prices can vary wildly. Do I have so much brand loyalty that I will pay perhaps 2x the cost? Like most people, I wouldn't.
In terms of availability and cost, LLM providers are much closer to Coke than to an airline.
If you are business traveler gaining status by flying a preferred airline and using other people’s money, you aren’t going to go to the cheapest airline.
Most of the profit from the Big three airlines come from business travel and credit cards
There are only two commercial flights a day, both on Delta and both to ATL. A round trip ticket is $540 for two 1 hour segments (MCO - ATL - ABY).
A round trip ticket from MCO (Orlando) to LAX (Los Angeles) is about the same price
Of course I know the trick for former - book through a partner AirFrance for 17K miles
It's in the article. Making coke is relatively easy compared to running an airline.
> Why should we consider LLMs to be more like one business or the other?
Also in the article. LLM's are analogous to airlines.
My specific point was that the article doesn't appear to support the assertions that it makes about brand loyalty.
It's a pretty bad guideline in my opinion but my opinion isn't worth shit here.
I'll re-read your comment when I have more time. Sorry if I missed the point.
The point of that guideline is to ensure that the conversation is substantive. Repeating points from the article with an assertion that those points are indeed in the article doesn't really add to the conversation and it's something that I do find frustrating on HN, which is why I mentioned it. I agree that it isn't a great guideline.
that should also apply to drinks, in which case coke should be underpriced by pepsi
and it should also happen to every other category of beverage
the author's argument falls apart as soon as you talk about something that's not the brand Coca Cola
"industry structure" should be about the industry, so this should mean other beverages like tea and water should also be strong industries?
but no they're not, they're terrible just like airlines
We know now how much actual competition[0] Google had after the dust had settled, in all practical terms - zero. Even after all the SEO spam and enshittification they haven't lost any notable market lead.
Time will tell if ChatGPT ends up that way but unless OAI implodes (which isn't all that unlikely) they're on the way there.
So being first does not necessarily mean winning.
And Twitter had strong network effects.
Many people started with AWS as it was first, and it leads to quite a bit of momentum in terms of market share long past when there was significant differentiators. It is just that there are switching costs and most people have already learnt AWS's APIs.
These are the big 3 so the only ones I mentioned. I know alibaba/yandex/digitalocean/etc exist but lack as much experience with them so only commented on the big 3.
That's not nothing, but switching costs are very low, and an alternative could arise faster than the switch from Friendster to Myspace or Myspace to Facebook.
As a product, ChatGPT + Python + web search + the interface are miles better than anything else except in some use cases I find Google’s NotebookLM to be a better product
Which is to say, the platform holders will determine who wins and loses. ChatGPT will win if they pay sufficient fealty to Microsoft, Google, and Apple.
It is:
- Dall-e for image generation
- a real time Python interpreter that can run Python code to answer relevant questions
- can search the web to retrieve and validate information
- has the infrastructure to handle processing at scale
At some point, they are going to have to rebrand. Which means they left a lot of first mover branding value on the table.
It's objectively a very bad crypto. It's the prototype and everything it does there is a coin that does it 100x better now.
But man, Bitcoin, that name has serious influence and staying power. It's a testament to the power of branding and being the first mover.
It's called the network effect, I believe.
Which means OpenAI really dropped the ball calling their first big success "ChatGPT". "Chat" was good, but three random (from the user's standpoint) letters? Ugh.
Whereas "Bitcoin" is practically Platonic. Branding platinum.
For many school kids I think it's all just "AI", not "ChatGPT".
Google also didn’t go through billions of dollars and was profitable when it IPOd.
DoorDash still isn’t profitable.
But either way, your argument suffers from survivorship bias. There are thousands of companies that fail and disappear into obscurity
The AVERAGE person still does not even know what ChatGPT is.
At most, 1 in 10 people have ever used ChatGPT.
This is like saying Social Networking is MySpace's to lose. Not really. Most people hadn't heard of Social Media or MySpace when MySpace was already huge and - by far - the biggest player.
It is likely easier for Facebook, Apple, Microsoft, or Google to introduce >50% of the population to an LLM than for ChatGPT to get from ~2.5% to >50%.
ChatGPT monthly users is about 1 in 40 people, by the way.
Does that mean ChatGPT is doomed to fail. No.
ChatGPT could easily be the winner.
But declaring the race over unless ChatGPT blows both its legs off seems very premature.
But it was. If MySpace had evolved, and stayed ahead of trends, and cannibalized their own products, and really understood the value of social networks... they could have leveraged their initial lead to a dominant position.
Saying the market is ChatGPT's to lose does not mean they can be lazy or incompetent or even just merely good. It means that all things equal, if OpenAI executes at an equal level to their competitors, ChatGPT will win.
It's like saying a marathon is the front-runner's race to lose. It's simply true. It does not mean the race is over.
Who is "the average person" here ?
How unknown do you think the site that got over 3.7B visits (#8 in internet worldwide traffic) last month is ?
The idea that chatgpt is this unknown thing doesn't hold up to any scrutiny at all.
The real business is enterprise API endpoint billed by the millions of tokens, and in that particular domain OpenAI has literally zero market lock-in (and they probably depends more on Microsoft sales power than on their own brand value).
Unless OpenAI can show that they are able to make money from the mass of casual users, they are in a tough spot.
Just because you can theoretically easily switch or that you brand has grown to the point of generic doesn't mean switching is going to happen. Habits are sticky and branding is incredibly powerful.
Anyone can use bing easily. In fact, bing is the default search engine on the default browser of the OS with by far the majority of users and stil...
It's funny because ChatGPT is such a bad mainstream branding. Technical name, hard to pronounce, nobody even knows what GPT stands for. They really got overwhelmed by their own success otherwise they would have done more on the branding side to appeal to mainstream users. But their first mover advantage won't last forever.
"[Trademark] Registration is refused because the applied-for mark merely describes a feature, function, or characteristic of applicant’s goods and services."
https://tsdr.uspto.gov/documentviewer?caseId=sn97733261&docI...
Branding is a moat but it's not a deep moat. Branding ironically works best (most profitablye) for incidental things that people exhibit to others - designer clothing is the most obvious - and this is because then brands have a social aspect (there's also branding a real signal of real superior quality - I'd buy a good brand of drill 'cause I have a rational reason to expect better quality but maintaining the quality of a branded product is more costly and hence less profitable than maintain the pure image of something like Coke and LLMs turn out not to really differentiate on quality). Whether they call LLMs "ChatGPTs" or not, people use LLMs for a result - they'll use a different LLM that gives equivalent result if they're motivated to do so. No one else is going to what brand of "ChatGPT" someone "drives", etc.
Let's do an opposite question. What's Google's moat? What's Apple's moat? All I hear from everyone is "X is not a moat", which while true doesn't mean company couldn't be ahead of the competition forever.
Apple isn't immediately seeking to compete in this field, presumably because they don't see a deep enough moat.
Apple's moat is people's hardware investment, their interface, their brand in a way that is socially significant as well as implying a real quality difference. Apple's overall moat is much larger than Google's.
Edit: and the specific non-moaty part of LLMs is that their answers are generic - LLMs don't have "personalities" because they are a trained average of all publicly available human language. If a given LLM had restrictions, it wouldn't be as useful.
> hardware investment, their interface, their brand
Exactly, you gave all the possible moats for LLMs. Not saying OpenAI has it right now, I am disagreeing with the premise that LLM provider can never have moat.
I would agree that moats are relative and companies can stay ahead without deep moats. But I think you still the problem of the specific way that LLMs are generic. Users don't invest in an LLM, they just learn to use them and that learning can transfer. User don't get "bragging rights" for using ChatGPT rather than a competitor (almost the opposite). ChatGPT output doesn't have a "flavor" distinct from other LLMs - in fact, as a user, I want the output flavor I ask for rather than anything identifiable as ChatGPT.
All LLMs are actually converging to about the same LLM, since they are trained on the same Internet/book/average-human-knowledge.
> habit
Habit matter for things people don't pay for. If a person pays, they'll go out of their way to get something for less. Microsoft's big thing is making sure end users never pay for Windows.
> integration,
Not going to matter. Every "AI application" is basically just a prompt and users can make their own prompts.
> hardware investment,
OpenAI doesn't even have a hardware investment, just a deal to use MS Azure. Other AI companies can and will just a cloud too.
> their interface,
Every LLM has the same interface. A chat window.
> their brand"
As above, brand matters for either habit (which again, only matters when thing cheap or free), social signaling (which a LLM choice won't give you) or actual differences in quality (which LLMs don't have).
By this logic every search engine should converge to same thing? Again I am not talking about current gen llm, just saying your assertion that the quality would remain converged forever isn't substantiated enough.
> Habit matter for things people don't pay for
This is so baseless and ridiculous. e.g. Excel/Adobe isn't ahead of competition for features.
> OpenAI doesn't even have a hardware investment
https://www.reuters.com/technology/artificial-intelligence/o...
Branding is an extraordinary moat in fact and it is very deep. That's why you can walk into CVS, Walgreens, Walmart, Target, Costco and buy Kleenex and pay a brand mark-up for it, and they have a lot of shelf space in most stores. For no great reason other than brand and people keep buying it - that magic branded paper - by the billions of dollars worth every year.
The same is true for cereal products. $6 for a tiny box of branded, sugar loaded, garbage cereal? Laughable, absurd, and yet people just keep buying it. $3 for a little can of soup, outrageous, and people just keep paying it just to get the brand. It's all for the brand.
The same goes for branded over the counter healthcare products, such as Advil, or countless cough & cold products and supplements. How is Advil still such a massive brand? The brand value is very, very deep. It is deeply entrenched into the consumer thinking process, so much so they commonly think Advil is meaningfully superior to generic labels.
The same is true for the sugar water of Coke and Pepsi. Or 5Hour Energy. Or Monster. Or RedBull. There is nothing particularly special about any of it other than branding + routine. The flavors are fairly easy to mimic or even surpass. Also goes for branded bottled water, most of which is silly labeling, the height of bullshit branding.
$45,000 - $60,000 for a middle tier metal shit box of a vehicle, from any number of the automakers in the bottom 90% in terms of quality. Consumers could go used for $15,000 - $20,000. Instead of piling up an extra $150,000 in net wealth over a couple decades, they do the really dumb thing instead, because they can't control themselves (extremely poor impulse control, same reason they're all so obese and unhappy). They buy those cars to keep up with their peers in lifestyle projection, same reason they buy the brands in anything. If you get position as a brand, you've got consumers in hand (then it's just down to fighting with the other brands).
There is some nuance to this. If you're building an application that embeds an LLM, then your "user" might be a prompt engineer, not necessarily the user using the application. It just so happens you can use the embedded magic using the prompt yourself.
Example: https://asana.com/product/ai
Not a single mention of Chat-GPT or Claude, but if you google you'll see they use Claude under the hood. So I would argue the branding is actually "AI" not ChatGPT.
It's a bit like Crypto and Bitcoin. Not all Crypto is bitcoin, but all bitcoin uses crypto to power it. People recognize both the branding of Crypto and Bitcoin.
Being the brand name of the industry is powerful, certainly, but it doesn't mean as much as it sounds like just based on usage numbers.
You're right that there are people who say GPT and mean something else but the vast majority of LLM users are actually using ChatGPT.
Yes, Nvidia GPUs are currently expensive. But they will soon be under tremendous competitive pressure from AMD, and more importantly Moore’s law is relentless (both in terms of model size capacity and performance per dollar). The price evolution of miniaturized transistors is basically the opposite of the airplane example.
Second, barriers to entry will keep increasing. Frontier models require stacking many new research and engineering insights. Of course the extent of secrets is currently limited because they only stopped publishing breakthroughs a couple years ago. Obviously that’s going to look very different 5 years from now.
On the other hand, competition between the leading frontier model companies is increasingly fierce (Google and Facebook have been slow to ramp up but theoretically should pose a big threat, and I am suddenly seeing Gemini topping leaderboards in the last few weeks), the moat is indeed questionable and the price of talent is very high. So it’s by no means an easy place to build a profitable business. But it’s at least possible for one or multiple of these firms to achieve process complexity that is extremely hard to replicate, and in the asymptote I really don’t think GPU costs will be a material threat to the business model.
Similarly, Google run all their stuff on TPUs as far as I understand it, and Microsoft and Amazon both have ML accelerators in the works slated for 2025.
> This is despite the fact that they are identical in both taste and colour.
The only way I could ever mistake Coca-Cola for Pepsi is if I were to completely lose my sense of taste. I'm quite surprised to encounter someone who considers them identical or interchangeable.
He's right, though, that I could change the supplier for airlines, browsers (or ChatGPT) and my world wouldn't be all that different. It would taste the same.
https://daily.jstor.org/the-coca-cola-wars-can-anybody-reall...
That's like drinking an unknown wine from a cheap plastic cup alone in an alley after a much too large breakfast, without any bottle label, price tag, companionship or comfortable context for proper tasting calibration!
Our senses really do cross over. Associations and history incontrovertibly shape our subjective experiences. Coke and Pepsi, in their naturally prominent can-habitats, taste very different.
My guess is those test ads were more interesting as stunts than as behavior changers.
As an aside, I don't think wine is necessarily the best comparison: the most recognizable brands of wine are often the least revered by connoisseurs . It's the experience of drinking wine from a glass bottle with a real cork out of a high quality wine glass in a classy setting that adds a lot to the flavor of wine. But which brand is relatively unimportant. That's not true for Coke where people's associations for "quality" are with the brand alone.
There is a fundamental difference in branding dynamics between markets that value novelty vs. predictability, and exclusivity vs. availability.
With the wine market varying across both those axes.
Then there is the consumption vs. collection aspect. Winos of all persuasions have individually varying degrees of immediate satisfaction, delayed-reward, and hoarding/treasuring instincts.
Not really a cola soda branding type market at all!
I'm quite sure I'd feel the same way on blind tests and I was assuming that most people have this keen sense when it comes to their favorite drink.
I've been commenting throughout the thread; forgot to say I used to work for Coke
The argument relies on the axiom that NVIDIA will have a persistent hardware advantage. Maybe they will, but even if they were always 2 years ahead of the competition, if NVIDIA-trained LLMs would 'good enough' in 2025, then non-NVIDIA-trained LLMs would be 'good enough' in 2027.
> NVIDIA will have a persistent hardware advantage
I think it probably has more to do with CUDA. The reason Python is the undisputed champion in AI and ML isn't because Python is a better, more performant programming language so much as because ecosystem of software in the AI/ML space is extremely concentrated, dense, and rich in the Python ecosystem compared to Java, Go, or C#.
Likewise, it seems like NVDAs advantage isn't even necessarily the hardware but the suite of tools and software that are built up to take advantage of that hardware.
Or maybe there's a highly profitable role for all the different parts of the value chain.
Not mentioned: a lot of the "money" that they raised, was not actually cash but credits for cloud compute. If you've already bought too much cloud capacity, giving away some of it and claiming it as an investment in AI, looks like a good idea.
Nvidia is currently important to training new LLMs but it's not that important to running inference on existing ones.
I think email might be a better comparison? If LLMs really do become something that everyone uses without thinking about (and at least anecdotally it's the first tech trend I've seen that all my non tech friends are using) then yes sure you can easily change provider but in reality most people are just going to use whoever wins, just like most people use Gmail.
So investors are putting in huge amounts of money to have part of the next Gmail, and many of them will lose but there will probably be some dominant player and sure you can change but once you've used one for a few years and it is as good as or equal to another, and it approximates to free, then you'll probably stick with it, compatible api and interface or not.
https://ashishb.net/all/llms-great-for-business-but-bad-busi...
Regulatory capture.
The incumbents can lobby for overly restrictive legislation based on copyright or "safety" concerns that make absurdly expensive for new entrants to enter the market.
Think about big tobacco.
The same relationship familiarity/fluency lock-in advantage of human personal assistants.
It’s incredibly difficult to start a new airline. Small and medium sized airlines are almost non existent. There is currently a pilot shortage. Most people stick with the same airline for the rewards.
Are you sure?
For example, in the future if you ask a model "are trans women real women" it will have to say "yes" in one country and "no" in another. In one country the LLM will have to talk about the "5000 years of Chinese history" and in another will have to say that's just an invention to feed their superiority complex.
The DOJ doesn't seem to realize this.