Hacker News Reader

Comment by hadlock | Hacker News Reader

If the price of the RTX 6000 (96gb blackwell gpu) drops to $2000 (original MSRP $8000, currently $16,000 usd due to "scarcity") then, all of a sudden, "we have ai at home" becomes real and why would you ever pay for a subscription for your company/family when you can run Qwen Flash Next (which runs great on consumer GPU via software like Strata) forever for free?

Engineering types and power users need not apply,but For Everyone Else, the "normies", Qwen Flash Next is probably "640k is enough for anyone" territory.

IPO, valuation etc depend heavily on frontier, or "good enough" AI to need to run on commercial grade GPUs. If Qwen Flash Next is good enough and runs on a $2000 GPU, why would you pay $20/30/100/200 per month in perpetuity?

I wouldn't pick Qwen Flash Next to add kernel namespacing and cgroups v2 to my custom OS kernel, but it will happily implement the Plan.md that Fable 5.2 wrote for a dollar.

View on HN

> Engineering types and power users need not apply,but For Everyone Else, the "normies", Qwen Flash Next is probably "640k is enough for anyone" territory.

Why would any of those non-power-users buy a $2000 GPU that doubles as a space heater (seriously, try sitting next to one during long runs in the summer) and thousands more for a computer to host it, plus all of the trouble of setting it up, instead of a $20/month subscription?

I do have the hardware to run Qwen3.8-Flash-Next at home, and I still heavily use my subscriptions even for simple tasks. The experience of using the apps on my phone on the go is great and I always have the option to scale the model up to the task.

The people who are going to pay thousands of dollars up front to avoid $20 are power users and people who have different priorities and motivations, like independence or ultimate privacy. GPU prices dropping won’t change that much.

Also, you don’t need an RTX6000 for Q3.8FN. There are much cheaper options right now at lower speeds.

> The experience of using the apps on my phone on the go is great and I always have the option to scale the model up to the task.

Just sidenote, this experience can be replicated with software like Paseo or some other options.

I usually have a frontier model orchestrating between dumber frontier, Chinese, and local models, using Paseo subagents, from my phone or whatever hardware I'm near.

Though I don't lean too much on the local models, I find them to simply be too stupid. They're fun for experiments though like quizzing them against Wikipedia or making them debate philosophy in loops.

Partly, it's much faster than the cloud models. Also, those cards aren't that hot, the Max-Qs are rated at 300W, and can be limited to 250W.

Tokens per second is slower on most local setups than cloud models.

We're talking about RTX Pro 6000s, local models on those are not slower than cloud models. With MTP, you're generally at 200-300 tps on one of the big MoE models in single stream in code generation (the small MTP models tend to be better at code token prediction than prose). The big cloud models are typically <100 tps.

Even if you own a RTX6000 anyway it's not obvious that it's cheaper to run Q3.8FN on it than buying tokens. Google tells me that it can produce ~1 million tokens per hour and uses 450W of power, which for me comes out to 18¢ per million tokens. That's very similar to the price on OpenRouter. Maybe during the winter when all the waste heat cuts your heating bill a bit?

The workstation edition defaults to 600 W, Max-Q to 300 W. 1M tokens/hr is 277 per second, which is roughly right for a single request at a time for a lot of models, not sure about Q3.8FN specifically, but these things scale really well to 16+ concurrents, I typically get 1-2k tps on Deepseek Flash when the cards are heavily loaded. MTP (a small network that predicts a few tokens which get checked in parallel against the full net, kind of like CPU branch prediction) makes a huge difference in single stream throughput.

it's probably not cheaper, but economics are only one concern at play

Bingo. It's always cheaper to rent...

I did similar math for my card (9070 XT) and it is close to break-even on electricity cost alone nevermind the amortized cost and requirement to have a desktop for it to go into. The parallelization and faster speed of the cloud makes it feel more expensive but it's just doing the same work more quickly.

The strongest case for local AI is for anything requiring privacy/discretion but I don't think individuals will collectively prioritize this. Convenience is king for the majority followed closely by cost. Businesses who care about protecting their competitive edge seem like the more likely candidates for local AI but only if ZDR providers aren't good enough.

Local AI will probably end up where it is now: in the hands of hobbyists and/or privacy-minded people who already own a GPU for gaming.

This assumes status quo. The privacy part will become more important as subscriptions become more invasive, ad-ridden, and hostile, as it always seems to go. Local will catch on as hardware availability improves and the community converges on common solutions.

Local models can still surprise us, a small 27B could be much stronger in 2-3 years than today, while capability to run it would exist on more devices.

> Why would any of those non-power-users buy a $2000 GPU that doubles as a space heater

Because, unlike the $20 Claude subscription, your own hardware will not respond "sorry, but I cannot do that" when you try to hack a device or game that you own.

Open weight models are also increasingly starting to deny requests, like recent Kimi showed.

Abliteration or even just different system prompts are generally enough to make this a non-issue.

That used to be true. But now the Chinese model companies are just not training their open weight sets on cybersecurity tasks: there’s no censorship to remove, it just doesn’t have that specific ability.

Moonshot reportedly removed some training data, which dinged its cybersecurity ability somewhat, but Kimi still has pretty strong ability - they still reported >80% on their xdev-bench-hard, down from 90% before the data removal. DS 4.1 is supposedly quite good for it (85%), and Z.ai has been talking about how they're trying to improve GLM's cybersecurity abilities, so it doesn't seem like a broad trend.

I hope you're right, but the word on the street is it's part of why Qwen is releasing v4 later than they originally planned. That's only a rumour though, so maybe its complete nonsense!

> instead of a $20/month subscription

And that $20 subscription isn't a good comparison. That gets you Astra or Opus. Better comparison is OpenAI's $8/month plan, which gets you Luna (which scores the same as Qwen on benchmarks). At that price, most of the cost isn't even the model; it's just hosting the infrastructure.

With all of the money these companies owe I don’t think that $20 or even $100 monthly subs will stick around forever, but who knows.

This is exactly correct imho.

The same thing happened with uber when they needed to start making money. It's only competition that stopped it from increasing more.

Ironically, going bankrupt would allow them to wipe out the debt, keep the GPUs, and offer cheaper subscriptions.

Who are we talking about? Pretty sure you’d have to liquidate your GPUs in this case

Same level of understanding as thinking going to the town square and screaming "I declare bankruptcy!" frees you from your debt.

I don't think so? A Chapter 11 is only possible if you can demonstrate clearly to the court that the underlying business would be viable without the debts. It also wipes out the investors.

AI startups are characterized by:

1) Market dumping. They are burning investor capital to buy market share.

2) A need to constantly raise money to pay for (1).

A Chapter 11 bankruptcy would make no sense in this situation. They haven't demonstrated that their business model is viable, so if a court ordered a Chapter 11 these firms would just immediately go bankrupt again, and that's usually a criteria to require a different and harsher kind of bankruptcy.

But even if for some reason a court did allow it, they'd still need to do emergency raises because their outgoings are so high. Debt has nothing to do with that, it's purely a matter of fixed costs and (in Anthropic's case at least) huge locked in contracts. But no investor is going to give money to a company that just permanently defaulted on them and ran to court to zero out the value of those shares. So they would be unable to raise and would immediately go bankrupt again.

Then you just move once they're more expensive. They have essentially no lock-in.

This is a crazy take. No one pays $200/m and uses a single agent at a time, and at $20/m it would take 100 months to pay back, and we don’t know whether frontier then will be bigger, similar, or smaller models yet.

The subscriptions (or rather, the model training) are subsidized at the moment, so all of this could change in a heartbeat. But e.g. Anthropic subscriptions seem to be terrific value AFAICT.

And that’s at $2000, which is never going to happen until these cards are several generations old. That’s probably close to or below cost.

I’m all for local AI, but comparing it to the subscriptions is not really a like-for-like comparison.

If you run it at home, it won't be realistically doing it 24/7. 8 hours per day is already 33% utilization. If the model is actually running half of that time, it's already 15%.

So, there will be always business for someone to host one in a data center, and sell the tokens at a markup, that will still be cheaper than buying something you will only use 15% of the time.

I don't think it's currently possible to calculate a reasonable valuation for an AI company using conventional techniques. It is too new, it is still rapidly evolving, nobody knows how it will reshape the industry. It's more of a poker game where the underwriters are betting on how much the investors are willing to throw into a perceived once-in-20-years oportunity.

There's a lot of use cases where you may not be personally interacting constantly but the LLM is running a lot of the time. A consumer-hostable model like qwen3.8 flash next take longer than opus or better models to complete software builds, and I set mine off before bed, before work, and over lunch to take the next stage of some feature. It is running pretty consistently, just over more time than if it ran on the more powerful datacenter hardware that we use at work.

The math still doesn't add up.

Let's say, you get 80 tokens/sec, so 80 * 3600 * 24 = ~7M tokens or $3.3/day at the current OpenRouter token pricing. That's ~2 years to break even at $2000.

That's also about 7kWh/day assuming 300W, so net $2/day or ~3 years. Given how fast the hardware changes, it makes no sense at all.

Also, in my experience, Qwen 3.8 heavily relies on reasoning to produce meaningful results. So where Gemini-Flash would just give you a usable result in like 5 seconds, Qwen would stumble on a trivial technicality (like completely forget '+' and '-' in a patch), then reason for a minute or two on "hey, but what about that requirement, maybe it is related"? and eventually come to the right answer. But the speed and the token use is abysmal.

I mean, it's a cool model and it's a big step, and it's crucial to have open-weight AI, but currently it's not practical beyond proof-of-concept.

BTW, the current subscription situation is silly. The vendor harnesses force you into excessive token use, and then the subscription gives you a massive discount compared to OpenRouter pricing. So it looks like a good deal. But if you get more structured with what you want the AI in each specific task, $100 on OpenRouter can get you surprisingly high mileage.

$200 / month is getting you access to frontier models that can't run on a $2000 GPU and let's you run lots of things in parallel. I'll very often run swarms of 10-20 agents that I couldn't possibly run locally without likely $100k-$1m of local compute?

Essentially is the same as "why use the Cloud" argument.

I could 100% see myself wanting local inference for some local use-cases: continuously monitoring my network for intrusions, checking my security cameras every few seconds, running long running video generation models, etc.

> Essentially is the same as "why use the Cloud" argument.

The case is much stronger. You can run pretty much every "cloud service" out there at home on hardware that is practically e-junk. It really does not take much to serve webpages, host email, or a fileserver. In all cases you get something better than a commercial solution.

And if you use your AI to set that up - because people complain the level of friction to keep such solutions working, then you can make your AI costs back in cloud costs.

If it drops to 2000, then it will take 100 months of $20/mo AI subscription to amortize out, not including the time value of money or the cost of running the machine. I don't think this is a good trade for most people.

I can’t argue with your math or conclusion, but the long term price will not be $20/mo when the companies need to start turning a profit.

And if they manage to build a moat the sub price will be even more.

$20 a month subscription won't get you very far with current models, but yeah buying hardware doesn't make sense financially unless you've got it utilized 24/7.

But this doesn’t include token usage? For $20/m there are serious caps on usage limits. I mean I can cap out the usage limit in about an hour and have to wait 4 hours to prompt again then there is the weekly limit. So with local usage your unbound on these time constraints but like others have said there is a power cost.

>Qwen Flash Next is probably "640k is enough for anyone" territory.

This is a weird comparison to make, because 640k is known mostly because of how wrong this prediction was. It turned out that people wanted more, and the more RAM was available the more RAM hungry applications were created. So probably the opposite of what you meant (that Qwen will be enough for most).

It was mostly a nod to the fact that most predictions about technology turn out being wrong, but at that point in time seem perfectly valid.

If RTX 6000 drops to $2000...

That's at or below cost. Nvidia would just stop making them. Of course used cards will reach that price ten years from now.

An RTX 6000 at $2000 is crazy talk. When was the last time the premium Nvidia workstation GPU retailed at $2000? Probably a decade or longer.

> why would you ever pay for a subscription for your company/family

For convenience mostly.

Why do the majority still continue paying for Netflix et al. when you can have your home media server at home with automatic downloads?

It's worth noting that media piracy is illegal and therefor complicated in most places, requires a lot of manual setup, and people remember seeing fines of tens or hundreds of thousands of dollars.

Installing ollama or lmstudio on a computer with a GPU is about as complicated as installing Netflix. Computers come from the store with GPUs installed and an OS.

Outside of dev people typically don't use LLMs on a desktop computer.

To be competitive with frontier offerings, it must be multidevice, available on any network, and integrated with all the apps people are used to use AI with.

Most tech enthusiasts don't self host super trivial things like a local DNS resolver, Their web presence, or email despite the huge incentives and the super low computing requirements, what makes you think they would build and run at least 12/24 a day a dedicated fan buzzing computer (to be useful for the whole family, it can't be the same as the gaming machine) in their home to be available for whatever request their family members may want to do to a model?

I known 0 "normies" that work on a desktop computer, or that would even remotely care to invest 2k upfront to not pay for ai subscription, or have a tiniest effort cpaacity to learn how to install it.

Why do you think everyone still pays for icloud or google drive instead of just self gosting a foss on nas at home? Hell, even me as an engineer know how to do it and know i would save money but i just can't be bothered because of convenience

Putting 2k in a 3% savings account already pays for three $20 months. Include the rest of the hardware around the gpu and electricity costs and you never save money by running your own hardware. Which makes perfect sense, since cloud providers have cheaper electricity and can use hardware more efficiently by running it at higher load. The reasons to run inference locally will never be economic. Perhaps we will reach the point where it becomes cheap enough not to matter so that nonmonetary considerations win.

Isn't the lesson of "640k is enough for anyone" that people will end up wanting more? Nobody today will buy a computer with 640KB of RAM, even though surely this would be cheaper.

I get quite all my work done with Claude Pro, which is around $20 a month, even using it at work. I could subscribe to this for 10 years until breaking even with your fictitious $2000 RTX 6000. Economically this just doesn't make sense.

> to add kernel namespacing and cgroups v2 to my custom OS kernel

To be fair, you wouldn't use an LLM for this.

I use LLMs for code review a lot, didn't try Qwen Flast Next but the bigger models do give better reviews so just having this case probably won't apply to me.

Sol 6.1 did it last night in two sessions, one each for namespaces and cgroups, I didn't even use max. Most of my benchmarks suffered, but not tremendously. I'm about 40% of the way through writing my own CRI to attach k3s to right now.

Curious why you want to use an LLM for such a task? I’ve built a hobby kernel before because it’s challenging, telling an LLM to implement a feature would get boring fast. Using LLMs at work is different, that’s often not challenging or fun… it’s work, but for hobby coding I just don’t get it.

If using an LLM is boring, then you aren't pushing what you can do with [local] LLMs enough or have an inventor's mindset.

I realized the other day that for my day job I only need about 10 apps; macos exists as a sheet of glass to run them on: a good terminal, vs code, chrome, codex/claude sdk, terraform, helm, kubectl, cpython, sops, nvim, email, slack, a generic windowing environment and a handful of other things (linters, templating utilities). Besides having a local container environment, as an SRE I am hoping to have this OS ready to daily drive as my primary work OS. I'm about six weeks in on one of these $200/mo plans and container support is the last feature I need to add before I start doing a performance tuning campaign of the process scheduler and memory handling. Adding these features by hand is interesting, but running my own completely custom OS on bare metal for work is way more satisfying. So far bare metal support is just raspberry pi 3/4/5 but I have risc-v working in an emulator.

> the "normies"

The normies are not going to pay $2000 upfront just to use AI lol. Why would they?

> add kernel namespacing and cgroups v2 to my custom OS kernel

deepseek can run locally on my strix halo and handles this kind of thing just fine. it really is good enough already for hands off programming, and hands on looping for slightly more complex stuff.

Which deepseek do you run?

at the very least where i come from, engineering types are those who most value privacy, and carry the greatest disdain for direct-to-customer saas in their own personal lives. diy ethic and that. QFN is as dumb as local is ever going to get, and you can run it at acceptable speed on a machine that gets good frames in a ue5 game on high and was built with enough ram for a virtual machine or two.

No one other than engineering types and power users want to run their own server. Everyone else just wants their phones and laptops to connect to the cloud and do the magic for them.

50 something SWE here. I’m happy with a 2015 MacBook Pro. Don’t need Ai at home. Ever. I think you have a misaligned view of the world.

Incredible what one can achieve with fantasy math and leaps in logic

Well the $200 plan will cost $20 instead because they have the same input costs, by your theory, or they don't, because energy isn't getting cheaper. Running your cluster at home in California can easily cost $900/mo

Flash next sucks compared to Opus 5.5 let alone fable

Engineering types should be plenty happy with this setup too

If it dropped to 2,000, I'd rebuild my server all over again with 384GB vram.

DSV4.1F is enough.

Maybe say "Qwen Flash Next" a few more times for emphasis.

You would need several RTX 6000 to run the large open weight models, and even then, the price of electricity would probably exceed API prices.

MoE next token prediction is higher than 80%; you can store the other experts in RAM and the engrams can be streamed off very mediocre nvme. It's been this way since September. I'm running it at home (although heavily quantized, there are limits) on 48gb ram and 12gb gpu. We run it at work on the 96GB RTX at a much better quant.

You aren't hitting almost 2x decode with speculation on a MoE model, but it's irrelevant. Electricity will cost more for most people than API will, even with free hardware. And 4-bit qwen next will be a major competitive disadvantage compared to larger models.

Qwen 3.8 is decent enough for half of what my family would ask it to do.

Reply on news.ycombinator.com