_davide_

for lack of directonality?

If compute is not the bottleneck, memory is easy-ish to produce (the hard part is mostly on the fab side); what stops a Chinese NVIDIA (huawei) from being 10x cheaper?

They are usually the same family, LPDDR is used for amd and macs, but the fabs are the same as the most expesive HBM memory, if they have a choice they are going to produce the ones that they can sell for more $$.

I'm writing my own inference engine for Strix Halo and the same model. I already have 30%+ performance plus a more graceful decay over long contexts; that said, their point stands: memory bandwidth is what you really want.

same experience here, as soon as it touched any gpu code it stopped working

> This is very literally what already happens, it's called a EULA. Yes, but they "reserve the right" to update whenever, making it pointless

> "In favor of the customer over anything else" is not a legally viable clause. I'm sure that legislators could put the principle down in a much clearer way. What's lacking is the will.

Yep, that's me. the only real blocker is that American companies don't trust Chinese providers, but i could just find a good American provider that hosts DeepSeek and/or GLM. I would at least be able to choose my own agent instead of a quite mediocre one that wastes time and output nonsense verbs in a pathetic attempt to gain sympathy. The only reason that stopped me from doing it is the absence of a subscription, and I did believe I couldn't get the same value with API pricing, but I'm starting to see that it's a blatant lie and true only for anthropic and openai...

A simple law: everything the customer buys must always behave *in favor of the customer over anything else*. If the product/service contradicts this, it must be fully stated before the purchase and cannot be updated. <= This would be a sane balance.

I'm tempted as well, just out of spite

It isn't a promotion, it's 2x the parameters of opus and we are paying with 2x the consumption rate. They just want to get rid of the subscription model.

"promotion" like they are doing you a favor just this once out of their goodwill... Really really really pissed me off

What a well written article!

just buy more RAM, it's cheap enough...

the threat is non-existing for agentic flows. Local interfere could catch up on high end phones

Sounds like a good solution my Führer

you can absolutely use it for some workloads, but as soon as you have some extra complexity for a big repo it'll take forever and the economics are so silly to the point that the electricity bill would be comparable to a subscription. I love having the possibility of running things locally if some random dude decide to pull them plug, and give me solice the fact that i can have 100% private inference, but as the main driver during the day? shoot me

i used to mix remote and local minimax 2.7(q3) on my strix halo, it run at 30 tg and 220 tokens pp... it was a bit painful slow, but it was a good feeling i could stay offline. unfortunately m3 which is at opus .8 levels is 460b parameters and doesn't even fit in 128gb of memory, let alone a big context. strix halo feels like a toy for ai purposes. https://kyuz0.github.io/amd-strix-halo-toolboxes/

I did develop my own agent around MiniMax. I did see weird behavior when I messed up the loop, like omitting pieces of remove thinking; maybe it's an agent bug, some models/providers just ignore/normalize the broken input, some don't.

>It's such a weird "Gotcha" that seems to only assume that Chinese LLMs might censor something.

We are not assuming anything; it is illegal, and you will get prison time just for talking about it. Yeah, sure, everyone distorts reality, but there is a huge gap between hiding and enforcing. So yeah, having models respond accordingly is unexpected. There are probably multiple variants tuned differently.

Consciousness doesn't exist, it's a vanity concept, to boost human ego...

I had a subscription before the price was cut down; the model kept randomly looping the with same character (burning 30% of the budget in one shot), and the overall performance for agentic purposes is, simply put, terrible. It finds non-existing bugs and randomly removes chunks of code to fix them, then even presents it as an "extra fix". Maybe it's a good generalistic model; I haven't tested it in that regard.

MiniMax (currently 2.7) which is a ~270B model tuned exclusively for agentic purposes, performs so MUCH better; it's more reliable and cheaper. Both are still far away from Opus 4.7 that I'm using at work. IMO benchmarks are just a very rough estimation; everyone cheats as much as they can get away with. Test the model yourself; do not make any assumptions based on the benchmarks.

I would love to see specialized, cheaper, bleeding-edge models like MiniMax for other non-agentic purposes as well. Why pay $1 for a general model when, for example, you can pay $0.1 for a content-moderator model that you actually need?

They do want to see the American bubble burst, this is the quickest way

100% agree, unwatchable and cheap, it's the most effective way to make sure I'll never touch the product.

Lol, use Gentoo

Except rust is way more productive than any other language out there...

Thank you, but no thanks

Most examples and presented issues would not compile or be a real issue... I stopped reading midway

> What could go wrong?

LoL, an insane amount of things. TCP connections are an illusion of safely, for the purpose of database commits use UDP packets as a model instead, it'll be much closer to reality.

I just got a Ryzen AI 395+ and compilation times are insane, kernel compiles in one minute! :D https://www.phoronix.com/review/amd-ryzen-ai-max-pro-395/5

I have been using Gentoo 10+ years, you can save time using other distros but you'll waste an insane amount of time as soon as you want to properly configure a yubikey or set up a modern compilation stack (or update specific drivers or fixing the kernel). Compiling everything seems like an overkill, but remember it's unattended time, just start the upgrade before dinner twice a week and you'll be good to go. (Failure to compile in Gentoo are extremely rare, less than once a year)

The upside is deciding exactly which packets you want and which versions, and whenever you want to customize( or fix) a specific thing it's trivial.