HNHacker News
TopNewBestAskShowJobs

kennywinker

11,615 karma · joined December 27, 2010

submissionscomments
kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
I mean, every person who pays them money helps them continue to do this to us. Part of responding is choosing who to give your money to.
kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
> on cheap hardware is just as good as the value you get from paying $200 to Anthropic

Not just as good. I’ve never said just as good. what i will stand by is good enough. What you can get for $600 is good enough to be useful for building software. Since paying anthropic/openai money to continue their sociopathic mission is a hard no-go for me, i’m just very glad there’s another option that passes “good enough* bar.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
> 25% of people will feel similarly enough to endure a meaningful drop in quality and intelligence in pursuit of those things you mentioned?

Anthropic continues to serve haiku - which suggests people have a use for less-than sota models.

A bugatti is undoubtedly a higher performance car than a honda civic. Yet if you're commuting to work in rush hour traffic, both cars will get you there in the same amount of time - and one is going to cost you a lot less.

The pitch of the frontier labs is that you need a bugatti for everything. I don't buy that.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
GLM5.3 costs $1.40/M on openrouter. 5.2M tokens per day is $7.28/day, which makes break-even take 83 days.

But let's compare apples to almost-apples! Qwen3.8 flash next is $0.47/M on openrouter, so $2.44 per day, so 246 days to break even. I say almost-apples because on this $600 system we're running an IQ3_S quantization. The hosted model is running... nobody knows what quant. It probably varies bases on the time of day.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
For what I do, that works. In exchange I get privacy, i get to not fund the big labs eating the world, I get to not worry about hitting plan limits, and I don't have to worry about them killing my account if I ask about the wrong thing.
kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Fair criticisms. If it's comparable with haiku, that's $13/day or 47 days til break-even (again, assuming I have my system generating tokens 24/7 - which I do not).

> You’re also comparing benchmarks for the full model

I wish there were more concrete ways to judge the difference. I've spent a bunch of time using 3 and 4 bit versions of qwen 27b, and my conclusion was that the lower qaunt made more mistakes - but that both systems were able to recover from those mistakes in a coding context. I'm not sure if that holds across the board, but that's the best answer I have so far.

It's also unclear if anthropic always serves you f16 versions of their models. My friends who use claude a lot complain about intelligence that varies from day to day - but that's entirely anecdotal.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Only if people continue to hire you… or pay for what you make. Hope so for you sake (genuinely!)
kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
I absolutely do have an ideological viewpoint. I hope i haven’t come across as trying to hide that.

One of my beliefs, call it ideology if you like, is that api cost is much more representative of real cost of this stuff. Good luck switch to local models the same week everyone else decides to switch because subscription prices went up 5x… it’s going to cost more than $600.

Anyway, like it or not, you are giving $200/month to fund another ideology. One that looks to me like a leopards-eating-faces type of ideology.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
> as little of me in the loop as possible

So not really work, then?

You’re automating yourself out of a job using capital you don’t own, and you think your position will hold? I am skeptical.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
… For now, until the machine you keep feeding eats your business.

Anyway, if it’s worth more than $200/month definitely don’t spend $600 and expect it’ll replace that. But $2400 ($200/month for one year) gets you much closer to sota than $600 - and whatever the line that stops being a rounding error for your business gets you closer still.

How much is it worth to own your livelihood? To me outsourcing to anthropic or openai for $200/month means you’re handing them the ability to kill your business at any moment without even noticing. That’s worth more to me than 30% points of intelligence.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Which is exactly why I compared sonnet 4.6 at the end there. They have similar intelligence scores.
kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
That chart is comparing the cost of buying tokens, not the cost of generating your own. The system I specced out can generate 5.2M tokens per day.

That's $104/day of opus 5.5 tokens, or $265/day of fable 5.1 tokens.

If you were able to saturate usage of this system, you could pay it off in 3 days of fable time or 6 days of opus 5.5.

But ok those are more intelligent. Let's go with something that scores similar - sonnet 4.6. 5.2M tokens is $78/day and the system is paid off in 8 days.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
I wasn't describing the best option, I was describing the cheapest option.

I know that configuration works, because I am running that configuration right now.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Astra and 5.5 came out in september, so I didn't compare to them I compared to "where the SOTA was less than a year ago".

But ok, let's compare: https://benchlm.ai/compare/gpt-6-astra-vs-qwen3-8-flash-next

Astra's score: 85.8/100

Qwen3.8-flash-next's score: 55.87/100

So there's a ~30 pt difference between them. Let's add 5% more for quantization damage.

My perspective is, yes - that's a huge difference. No - it doesn't matter. Flash-next is above the threshold of useful intelligence. I can ask it to do something and it does it. Sure, it burns some tokens on bad ideas on the way, and sure it can't turn a single paragraph prompt into million dollar business. I don't need or want that - I want something I can ask to refactor a module, and it does it and it doesn't stop until the module compiles and all the tests pass. flash-next does that.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
> there’s no world where “normies” can do this.

Not today. But when it turns into a $600 brick you plug in next to your router, that's gonna work for most people.

For today, I'm speaking to software developers - who can 100% handle setting up a local AI system - it's only as hard as building a PC.

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
The floor for a functional local AI system keeps getting lower, dispite ram and GPU prices continuing to go up. An E5 workstation with lots of threads, $300 of RAM and at least $100 of GPU gets you something that can do >100tok/s+ prompt processing, and ~20tok/s output with Qwen3.8-flash-next:IQ3_S. Toss a $500 GPU in there, and it goes from barely usable to pleasingly performant. For less than $800 you have something that holds its own with where the SOTA was less than a year ago.

I don't see how anyone justifies a claude subscription. Do you want to rent the tools you use to making a living from the people who are trying to automate away your job?

kennywinker··on Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Try strata. I’m not sure how well it handles igpus but if it does i’m pretty sure that system will go screaming fast
kennywinker··on Upload Your RAM
cloud? I feel like that's an example of things going the other way. It used to be every company would have either on on-prem or a colocated server at the nearest network hub - a fairly de-centralized system, and then everything went centralized with cloud platforms.

To be clear, I am not skeptical that it will work this way - I think it's pretty likely llm usage will be come a split of de-centralized and centralized. But I am not very convinced this is a natural law of technology.

kennywinker··on Ask HN: How did you convince family and friends to join your XMPP/Matrix server?
Just a little shout out to chatto (https://chatto.run/), which is the best slack/discord alternative i've seen so far. Not federated or crypto based, but just a solid chat platform.
kennywinker··on Upload Your RAM
Can you think of examples of this cycle? I know this is roughly what happened with mainframes, but I struggle to think of other times it went like that.

Maybe arcade games?

kennywinker··on AI is dramatically accelerating AI research
vi research is dramatically accelerating research into how to quit vi
kennywinker··on AI is dramatically accelerating AI research
> People don't get how serious this is. If AI doubles revenue but drives silicon valley firms' tax rates from 5% to 10%, it's a loss.

Would you rather make $1,000,000 and pay $50,000 in taxes, or make $2,000,000 and pay $200,000 in taxes?

Unless you’re terrible at math, i feel like i’m missing something ;)

kennywinker··on Ripple-65B Preview: U.S. Open-Weight SOTA on Agentic Coding
All marketing relies on some degree of deception.
kennywinker··on Ripple-65B Preview: U.S. Open-Weight SOTA on Agentic Coding
In the absence of a clear reason to bring up race, the reader is left to assume the worst
kennywinker··on Ripple-65B Preview: U.S. Open-Weight SOTA on Agentic Coding
Why is the race of the people working there an issue?

I understand why some people want to differentiate between models from chinese companies and us companies and eu companies etc, because that is about control over the company. The us or chinese gov could, for example, make it difficult for canadians using a model under their control. Even with open weight, these could be weaponized to some degree. Arguably that gov control can also manifest in the model's behaviour, but since I'm mostly using them for agentic coding not political advice I don't really care about that.

But the race of the workers making this stuff... I'm not less concerned about a model made by americans working for a chinese lab than a model made by chinese people working for a chinese lab. If the race of the people building the model matters to you, why?

kennywinker··on Ripple-65B Preview: U.S. Open-Weight SOTA on Agentic Coding
State of the Area
kennywinker··on Ripple-65B Preview: U.S. Open-Weight SOTA on Agentic Coding
Which you're welcome to do and then people are welcome to disagree that is an important frontier to consider.
kennywinker··on Show HN: Subvocal – Talk to your computer without making noise
It sure is annoying when people respond to your behavior, I guess…
kennywinker··on Ask HN: Which TV do you recommend for privacy?
The issue was BTLE not wifi... and BTLE antennas are sometimes just circuit board traces, I believe.

Anyway, any solution that involves opening up the enclosure is one just for the nerds.

kennywinker··on Show HN: Subvocal – Talk to your computer without making noise
> Meta Ray Bans and even though I love the functionality

Which functionality do you love? Is it the nonconsensual recording? Or maybe the giving away your own privacy and the privacy of everyone around you to one of history's least-trustworthy companies?

← PreviousPage 2 of 34Next →