Sam Altman Thinks You're Wasting DRAM | Hacker News Reader

Sam Altman Thinks You're Wasting DRAM

frvr.com

106 pointsdutchbrit116 comments

Two statements can be true and he stated one of them. Sam is not wrong.

1. Local inference can cause a large under-utilization of resources from users that do not run local inference hardware at capacity over the hardwares lifespan. This is under-utilization and under-utilization is not optimal from an economics perspective.

2. The concentration of inference hardware to singular providers can aggregate power to single systems or organizations, and their incentives may be at odds with individual users.

The same power dynamic and economics struggle has existed nearly everywhere for all of history. For example, if you want to decide to own or rent a piece of heavy construction machinery for business, you will find a long list of do-not's on the rental contractual terms, but you are more than welcome to buy your own and operate your own machinery at several times the price for the hours used and use it however you want.

Few people will argue with the logic of whether it is technically accurate ( assuming actual deployment will be optimal, because that is hardly a given ). It is just amusingly self-serving. It is almost as if McD got to decide you can only eat there.

"Few people will argue with the logic of whether it is technically accurate"

Must why people in this HN thread are throwing out so many ad-hominems instead.

I mean, that's assuming we're aligned on goals. I don't know that Altman et al. care much about digital sovereignty. I don't think it's ad hominem to debate the validity of someone's ethics or motivations.

Besides, some things are done for reasons other than economic efficiency. It would be more efficient for people to relinquish their apartments to some company that will ensure near 100% occupancy while you aren't home. Yet you would be hard pressed to find a single human being who would rather share their space like this than own and manage their own condo (apart from those who get scammed by timeshares).

Humans simply don't work like that.

I would like to know how “efficient” Altman's $27 million mansion is.

Weirdly, underutilization of hardware was never a problem before. In fact, wastefulness is so ingrained in programming culture that the previous hype - blockchain - was about an explicit race over who could waste the most computing power on nonsensical calculations.

Also, I'd be very surprised if the current rush to build datacenters everywhere is optimal usage of resources. How many companies are building datacenter commons and sharing with other companies to optimize resource usage?

> Two statements can be true

As I general point, because I keep seeing this line on HN: Of course they can in the general case, but more often than not, the reason why this line is written is because there is some tension or contradiction between the statements. In that case, understanding where the tension lies is IMO more useful than shrugging and deciding to "hold two contradictory thoughts in your head" like in 1984.

Sometimes it feels like a rhetoric device to "rescue" an opinion when the opposite opinion is hard to argue against.

> ... blockchain - was about an explicit race over who could waste the most computing power on nonsensical calculations.

A hard rule of life is, if there are resources available, something will consume them. It's interesting that this also ended up being true for compute!

This is true, but I think building an entire technology on top of the question of who can consume the resource as inefficiently as possible is still something else.

This is why fusion power is scary. Once invented, we WILL irreversibly convert all seawater to computation as quickly as we can.

99.67% of seawater can't be used for fusion. Some very poor napkin math, that's around 1.3 million cubic miles, with around 2200 cubic miles per inch of sea level, which is around 600 inches, or 50 feet drop in sea level if fully depleted (not including any sea level rise from global warming).

Underutilization was indeed a problem before. The anecdotes you've presented are not broadly applicable.

The entire reason SETI@Home started was because people had unused hardware sitting idle at home. Every company I've ever been in has had slides on resource utilization for procured hardware.

In fact not only could the two statements be true, in fact both are true. People want to ignore otherwise because doing so will make local LLM inference sound like the only option when only one is considered.

Nobody put "your computer isn't idle sometimes" as a problem, or shamed users for it.

What they did was to say "hey, we can leverage your idle computer to help process this".

Different perspective.

Nobody has shamed users for having unused compute, only stating that it is not economically optimal when there are global restrictions on a resource.

It can still be optimal because some activity A might provide more utility at 10% utilization rate than activity B at 100% utilization rate . Otherwise shared cars would be vastly superior to individual owned cars and same for houses in which 2 or 3 families live in different shifts ... we actually had that before in some areas in early stages of the industrialisation.

Users apparently don't need to be thought about at all if a few companies can secure loans large enough to purchase all dram capacity, and in the process close that market for consumers.

It seems to me it's economically optimal for inference providers to intentionally close markets that would allow local inference at all, even if that's bad for users as a whole.

If those companies can pay for those loans from the DRAM facilitated revenue then it will have been an efficient use of resources.

And if they're wrong, you'll be able to purchase second hand cheap data centre grade hardware in a couple of years and DRAM prices will return to normal.

If you, or I, or a company want hardware we can choose to pay market prices, but 4+ years of students will be robbed of the opportunity to tinker with a class of hardware that would have been beneficial unless they have extraordinary resources or an extraordinary support network.

There is opportunity theft here.

>If those companies can pay for those loans from the DRAM facilitated revenue then it will have been an efficient use of resources.

"Efficient use of resources" is not some absolute based on mere energy or economics calculations, it depends on the goal.

If the goal is "reduce reliance on big tech", "privacy", "autonomy", and so on, then the most efficient use of resources if to say fuck you to Big AI and not let them hoard ram and gpus.

>Weirdly, underutilization of hardware was never a problem before. In fact, wastefulness is so ingrained in programming culture that the previous hype - blockchain - was about an explicit race over who could waste the most computing power on nonsensical calculations.

Same with AI. AI use is a colossal waste, for all kinds of frivolous purposes (e.g. some person's ai girfriend chat, or someone asking about K-pop lore, or slop image generation, or slop "dark internet posts"), but also to facilitate "serious" business that we'd be better if it wasn't done at all.

> Weirdly, underutilization of hardware was never a problem before.

Fixing exactly this has been the constant drumbeat of IT since the early 00's at least. It started with virtualization, to get capacity factor of hardware up in the enterprise. The average server back before VMWare (and HyperV/etc.) became a major thing was probably sitting at 10% utilization or less 95% of the time. Hypervisors were invented to fix this as their primary use case. You'd take a 5-10% performance hit on the raw server performance to gain 90% efficiency back.

Next came the cloud. Which was virtualization at scale. The larger the scale, the better the efficiency.

It's basically almost all anyone has actually accomplished in IT over the past decade or two. Everything else has been largely stagnant compared to the 80's and 90's.

AI inference is in it's infancy, but I imagine we will see more or less the exact same thing play out yet again. Lots of initial on-prem stuff due to cost and freedom, and eventually it becomes virtualized, commoditized, customer segregated, etc. Just like average cloud workloads are today and all the various "small" megawatt-scale builds moved back into the hyperscaler type providers within the next 5-10 years.

> How many companies are building datacenter commons and sharing with other companies to optimize resource usage?

Quite a few that I've seen. Rented inference/token factories are the current new hotness. Plenty of speculative builds being done by hundreds of smalltime providers. These will consolidate per usual and everyone but the last few top percent will remain as the winners. Just like the first hosting providers. Virtualization and fancy schedulers to maximize hardware utilization across a number of end-user customers are a huge topic in this space at the moment. Heck, a lot of these small time players are providing raw compute for the giant AI labs and even hyperscalers.

I don't like that Altman is right on this btw. I much preferred the highly inefficient days of every midsize corporate office having a 200-500kw datacenter in the basement. AWS and their ilk are quite obviously orders of magnitude more hardware efficient though.

OK, but Altman is complaining about individual users. Nobody said that maxing out my PC for gaming purposes or just to keep Windows from getting sluggish was a waste, pretty much ever.

And, "waste" has an overtone of moral wrong, of "you shouldn't". When before did anyone say that I shouldn't get a nice PC? Never.

So, yeah, moralizing to individual users that they shouldn't buy DRAM so someone else can have it is new.

He doesn't say anything about individual users. It's left unclear exactly what he means when he talks about low utilization inference using DRAM. I figured he was talking about companies building clusters. I see way more people using old GPUs for local inference than buying 15-20k Mac Studios, or RTX Sparks. I've seen plenty of stories about how companies are building clusters because they want to run open source models, partly for privacy purposes, partly because they're afraid of a rug pull, partly because they want to control when they move to new models, etc, etc.

> He doesn't say anything about individual users. It's left unclear exactly what he means when he talks about low utilization inference using DRAM. I figured he was talking about companies building clusters.

No, he absolutely does:

"From a global allocation perspective, actually, it's quite terrible to have everyone have a bunch of the world's limited supply of DRAM sitting in their own computer. And be used 5% of the time," Altman said.

> "How many companies are building datacenter commons and sharing with other companies to optimize resource usage?"

They're all sharing with each other. Anthropic's S-1, for instance, shows $100 billion+ commitments each to renting datacenter space from AWS, xAI, and Google (two of whom compete with Anthropic on AI). None of theme are idly sitting on unused depreciating hardware for no reason.

I've never heard anyone accuse FAANG's of not trying hard enough to optimize their own resource usage.

There was indeed a lot of blockchain bullshit, but Bitcoin had and still has its usefulness. Though not necessarily for you and many others.

"Local inference can cause a large under-utilization of resources" ... did he mean the resources that users themselves have fully paid-for themselves?

Not to mention, I bet users will be open to "rent" their hardware for proper utilization, provided he can pay for it.... real question is, will he?

> This is under-utilization and under-utilization is not optimal from an economics perspective.

The problem is that this sneaks in a postulate by negating "optimal", which is that underutilization is the opposite of that which is desirable, that it is implicitly undesirable. If we don't take that premise, say that it's value neutral and neither desirable nor undesirable by itself, then the argument doesn't really stand. This neutral valuation is far more reasonable to me. Anybody who works in logistics intuitively already understands this. We don't have cargo ships bow to stern across every single shipping lane, yet this is "underutilization" of those shipping lanes. Sure they're bottlenecked at the ports, but the ports can always be bigger, more numerous and possibly even work faster. The slop turns out doesn't matter. Actually it's beneficial for a few reasons, but my point here is sufficient I think.

You're right this is ultimately a power struggle at the core of it, what I think you could stand to acknowledge is that's all it is. It's just dressed up as economic rhetoric.

he forgets humans also use RAM for other things than inference -_-, his logic is not wrong but hes mismanaged his context window :p

also, its a bit weird to say a supply chain fails if you simply order much more than they have.

in stock:100 'i want a million, the supply chain is failing and others need to stop buying so i can get a million'

I agree with him. And I'll gladly donate my 128GB of DDR5 if it means OpenAI and Anthropic will be nationalized to ensure equal access to resources. Central planning will surely create the most efficient system

3. Some applications are going to work just fine with smaller models on local hardware. If people still have machines with the memory to run those models laying around, that will cost OpenAI business. Also, once RAM suppliers catch up and Moore's law resumes operation, the capability of local hardware is going to steadily improve. Were it not for the AI build-out, typical configurations for new PC's would likely have moved on from 32GB by now.

4. Everything is training data. If that data isn't going to OpenAI, they're not training on it.

Also, it's important to remember OpenAI's failure might actually be the best thing for everyone else in the AI field. e.g. Some new innovation/approach could come out of left field and eat OpenAI's lunch. They've bet the farm (and a lot of other peoples' farms) on a brute force approach. Altman needs your RAM because, if he doesn't control the brute force, the brute force approach might not work only for him.

Hold on a second. #1 contains a diverging value judgement rather than just a description of reality. Its truth value varies wildly depending on who's reading it. It is only under-utilization by Sam Altman's definition of utility, but not my definition of utility. I want hardware to be sitting idle in people's homes most of the time instead of processing requests in a datacenter (in large part because #2 is true). That is positive utlity for me. The same cannot be said for Sam Altman. We do not have the same utility function.

#2 is true for almost all observers. #1 is only true for a small group of people. #2 is an "is". #1 is a "should". Sam Altman is very, very wrong in the "should" sense of "wrong", by my judgement of value, and I suspect also by the judgement of most people reading this.

> 1. Local inference can cause a large under-utilization of resources from users that do not run local inference hardware at capacity over the hardwares lifespan. This is under-utilization and under-utilization is not optimal from an economics perspective.

Under utilization is not optimal only in one perspective. Maximum utilization of resources. That does not mean it’s not optimal economically.

This kind of underutilization has many other economic benefits. - Decentralization - Separation of ownership - Redundancy - Hedging against certain risks (such as the internet going down)

Economic value is more than just maximum utilization in the minimum amount of time.

Guy with the most DRAM in the world thinks it's terrible he doesn't have all of it.

It would be more efficient to ban cars in favor of mass transit. It would be more efficient if people only lived in apartment complexes rather than standalone houses. Tons of things can be more "efficient" if you take away people's desire for privacy and ownership.

Mass transit and dense urban environments are actually pretty cool if done rightish, as seen in Europe. I'd still want both options to be available.

> Guy with the most DRAM in the world

Citation needed.

He's right in a such a narrow space on this particular matter that it's just a perfect example of what's wrong with a lot of things lately.

OpenAI triggered this ordeal with its order of 900K DRAM wafers per month on october 2025, making everyone panic because it was around 40% of the world production at the time.

We all know that memory providers know what near bankruptcy feels like when they over invest, but right now SK Hynix have 80% margins (ie TSMC ~60%) and choose to pay their shareholders this year the quivalent of ALL their revenues in 2025. It's now common knowledge thanks to Samsung that the rate of factory buidling is not maxmium to portect their margins. And memory providers have already been fined in 2000 for price fixing.

Supply is short because they all choose to make it that way.

Seems like micron is pulling out all the stops to get their new line open. Source is someone who lives near the micron plant and talked about their insane hiring and work right now.

Everything that has happened is 100% aligned with ECON 101.

Demand has increased for high-bandwidth memory. That increases prices and quantity.

The increase in quantity has happened. Quantity sold has increased. Fabs have increased production and shifted away from other products including consumer products.

Prices increase when there is an increase in price of a substitute in production.

New fabs are planned, consistent with profit-maximizing behavior increasing production in response to increased prices, but they take time.

This reminds me of something I've been curious about.

When e.g. memory manufacturers don't collude, they end up on synchronized boom-and-bust cycles.

But it's also illegal for them for form a cartel, which IIUC would have the benefit of smoothing the supply:demand ratio over time.

Is there any truly good solution to this?

(I'm not an economist, so it's quite possible that my question isn't even cogent.)

Solution: Don't become too dependent on small number of customers.

... or save some from the boom so you can survive the bust until the next boom, (AKA being financially responsible).

> Is there any truly good solution to this?

Yes, what smart RAM manufacturers (or who've been longer in the business) are doing: ramp up production capacity, but do so gradually.

Those ramp-up timelines tend to be longer than the markets' boom & bust cycles. So they'll be looking at longER-term trends to decide what to invest in & when.

Which might be what is happening here: they worry (or worried? supposedly Micron is building a new factory?) this AI boom will/would go bust soon, so (too much) investment in new capacity might not be worth it?

[flagged]

I love when people say entire ass instead of just ass.

This suggests there are tiers to ass. What dictates how much ass someone can eat? Is there anyone who can eat only partial ass?

> What dictates how much ass someone can eat?

90% perspiration, 10% inspiration.

he might have a donkey, though that's rotten to say if so.

Maybe GP doesn’t like his fractional ass eaten.

All well and good for now until the free lunch of free models ends, and the ones you can "freely" download have weights that tell you to drink Ovaltine or whatever other messaging gets sold.

Abliteration and fine-tuning remain standard operations on open weight models.

will they ssh into my computer and edit the models i have downloaded in the past?

Will you be happy when those models are five years old? Ten years? They'll age out in their ability and their cutoff date.

If the tech never gets any better, I already have the power at home to re-create almost any software that exists.

Also home training and torrenting are a thing now too. Open weight models will never stop being released one way or the other.

This is an ironic comment, because my read is that open weights are literally the ONLY place this isn't going to happen.

mainly because the thing about open weights is folks will just immediately strip it back out.

But I can guarantee you that openai/anthropic/other will absolutely jump at the chance to monetize users by feeding them paid recommendations.

Exactly, just like the free operating system known as Linux is famously full of ads, if you don't like ads you have to use Windows

> Sam Altman can eat my entire ass.

Can you please not do this here? It's not what HN is for, and destroys what it is for.

https://news.ycombinator.com/newsguidelines.html

Fair Enough. Edited to make the same point in a way that I believe complies.

Altman: You are welcome to invest a little bit of your fortune in making major software developers write more efficient code that stops wasting our RAM. Start with internet browsers and the Windows 11 app that wastes one gigabyte showing the weather.

Most of the RAM & CPU resources used by web browsers are mainly the fault of web developers who have no clue about memory management and will happily write code that uses 100% of a CPU core to do nothing.

The vast majority of websites could be plain text, throw out the scripts.

That's mostly true. If you use a script blocker, most websites work fine, and the ones that don't usually only need one or two scripts enabled out of dozens of running blocked scripts.

In a sense, I think everyone knew it was coming. Historically speaking, in US at least, there initially may be a notable number of upstarts, but those numbers quickly dwindle to form oligopolies. That is how we end up with 3 million cereal types, but 2-3 companies controlling them. The same repeats throughout most industries with an annoying consistency.

I actually use both gpt and local qwen and I personally think the question tends to be of what one needs ( speed? privacy? lack of corporate silliness? ).

Still, it is sad that we can predict the future so well based merely on what happened before.

Edit: I certainly hope I don't have to explain to you why the above is relevant. No market participant would dare to argue this if the competition was healthy.

He could always just rent PCs usage time from anyone if he wants more RAM. Maybe using a sort of "token" system to calculate the remuneration?

i think Sam Altman is wasting oxygen

Selfish of him to keep all those nutrients locked up in his flesh prison when there's a climate-induced global food shortage coming. He feeds but one now, he could feed dozens - perhaps thousands, if rendered into fertilizer.

Was thinking of selling my desktop because I wasn't really using the compute but now I'm doubling down and using it for local AI inference.

I think Sam Altman is wasting space - as in he needs to get the hell off this planet.

I use my RAM how the hell I want and nobody else better damn well think about telling me otherwise.

The look on his face looks a lot like Elizabeth Holmes in You Can See Everything

He said he would never decive you though

Don’t you worry Sam, there will be plenty more of DRAM for me to underuse once ClosedSI goes belly up.

I mean, he's right: individuals hoarding scarce resources is less efficient that everyone holding them in common. The problem is, how to ensure we all get a fair share of them. We can't leave that up to commercial enterprises, because they also have an incentive to hoard and control them for their own profit. Perhaps what we need is some kind of government that holds them in common for the benefit of the entire populace and. But what to call it? Common-ism...?

The problem is he has no desire for resources to be common assets, he wants them to be his assets

Yeah. Well, forgive us for failing to make that our priority.

"I mean, he's right: individuals hoarding scarce resources is less efficient that everyone holding them in common"

I absolutely agree, we should redistribute the wealthness of those hyper extremely wealthy people back to society.

Think of all the silicon that could be harvested from private beaches.

And who Will use their Services without ram in their pcs/smartphones?

It would be interesting if there were some kind of holographic LLM. You could send part of the encoded prompt and receive generated tokens that only you can decode. Then we could use the distributed capacity of the world and everyone could be paid for marginal token generation capacity. Even open models are profitable to run on almost all electricity.

Leave the general computing to us, citizen.

The primary source is

https://cheekypint.transistor.fm/39/transcript#:~:text=quite...

Sam says a lot of things on podcasts, see https://sputnikintelligence.com/people/zlsl-zr6t-7llx

Why has this been flagged?

HN doesn't have twitters notes system, so flagging is the only way.

The linked interview in YouTube has sama saying basically what the article states, you can go and watch it. And I think it sparks a discussion worth having, here on HN.

Sama flagged it

How? I'm still on ddr3.

The most fascinating thing about this is how american billionaires can speak in public and advocate for abolition of private [computing] property (means of production), to give full control over these resources to a handful of megacorporations (the party), in effect letting them control and command the economy (central planning). It's the communist boogeyman wearing a trench coat under the banner of capitalism!

I understand what he's saying. Sam Altman thinks that HE, with his data centers, will make better use of the potential of VRAM (or whatever word designates computing capacity) compared to the local market.

In theory, I don't disagree. He needs it, and he will use it, that's for sure. Moreover, his logical argument should be viewed purely through the lens of his own self-interest: "I need it, I don't have it, it's your fault, and anyway, you don't even use it properly."

But the question I ask myself is: can he use VRAM for something other than what is requested millions of times? "GPT, make me an app to boil eggs," "GPT, make me an app to compare prices," "GPT, make me an app to alert me during Black Friday..."

So many resources used thousands of times... With his VRAM... So the counter-argument we can give him is: Stop thinking like an American picking flowers with a bulldozer (no offense, it's just a metaphor) and learn to conserve your resources.

I have a vehicle in my driveway, but I am not driving it all day, therefore I am wasting it and should give it to a corporation to rent it back to me. You will own nothing and be happy

It's terrible that all the unused money in these pools of really rich people is being wasted, it would be better if we could centralize it in the goverment (for example via taxes, the 1980 style taxes) so it can be put to better use.

by the way, why is the post being flagged?

Get off my lawn.

People shouldn't own books or printers either

Maybe what's really inefficient is sam a and other billionaires unilaterally deciding to capitalize and productize technologies imminently slated to get 100x more energy efficient, rather than distributing hardware in a way that would surely motivate and accelerate efficiency breakthroughs, even if more slowly.

RAM shortage is engineered and he confirms that.

He is obviously correct about the specific narrow statement made here and it’s so annoying everyone has to be memey mad about it like a bad reddit thread.

The part people take issue with is him saying it’s “terrible” not the assertion that it’s suboptimal in economic terms

Yeah, this. "Terrible" implies a framework of better and worse, and his axes don't align with everyone else's.

And even in economic terms... the way free markets work is that we let dollars decide which resources go where. That's the "optimal" distribution of scarce resources - they go wherever most values them. So his distribution is in fact economically sub-optimal. I value having some RAM more than he values having one more increment of RAM, so I get some and he doesn't.

He doesn't like it? Not my problem. Not economic theory's problem, either.

Right, he should have phrased this differently, but everyone is ignoring that he explicitly says "from a global allocation perspective" immediately before it, making it clear the "terrible" part is from a very narrow perspective, not as s general rule.

The "terrible" is a misleading quote of him making the assertion that it’s suboptimal in economic terms, it's not okay to chop up what he said and act like he said something different.

Sam Alt-RAM

Its terrible to have a few people have a bunch of the world’s limited supply of wealth instead of shoving it all in everyone's bank account.

In a way he’s right, but it’s not my fault a lot of the software i use is some web bullshit that ships it’s own chromium runtime.

No he isn't. Most AI "utilization" is from idiots asking stupid stuff and doing stupid projects. They have all the resources they need for serious stuff. Fuck them.

Yeah seriously how much of his ram is gonna end up going to AI cat videos and mentally ill losers who fell in love with the computer.

> Most AI "utilization" is from idiots asking stupid stuff and doing stupid projects.

ah yes, the classical elitarist gatekeeping. god forbid laymen and paesants can have access to technology! only a small caste of technology priests must be able to use the new technology!

Nope. No gatekeeping. I have been the exact opposite all my life and you could not be more wrong. But it's true that most AI usage is not by people who seek to learn and grow, no matter their background. It's just morons. So fuck them and fuck Sam Altman saying we have a "shared interest".

Edit: To be fair, I could have worded my earlier comment differently.

There is a huge loophole in the reasoning: Sam Altman assumes, without stating it clearly, that RAM is used only for AI use, and the only question is "should AI inference being run in the cloud or on our computer".

But that's not true. Computers, including RAM, are used for many different things that do not involve AI at all. video games, office work, communications, access to culture, etc.

So, what Altman really says is "AI is the only thing that matter for computing, and thus, for the success of his company, we must allocate all the RAM to his company, and forbid any people from accessing it".

And, IMO, we should not accept that kind of discourse :s

Reply on news.ycombinator.com