ekidd

Also, a word to the wise:

The reality of running recent Chinese open weight models is that they are all overtuned for successfully completing tasks. I can ask Qwen3.8 Flash Next to grep for regexes that use PHP-specific features that might be problematic to port to Rust. But instead of just grepping, Qwen3.8 FN will spend 30,000 tokens writing a Rust test harness, exhaustively check every regex, consult a bunch of docs, and write me a 2 page report. Q3.8 FN is deeply convinced that it's going to get extra credit for doing more than it is asked. (To be fair, it's a prototype release.)

But if your sandboxing isn't set up to handle some whacky weirdness, then you can't even trust Claude or ChatGPT. Which is important—the big vendors often do provide sandboxing or command filtering in their harnesses, but some popular local harnesses like Pi.dev are intended to rely on external sandboxing tools.

Yeah, this particular example was pretty sketchy on Cohere's part. Unsharing a namespace like that is a pretty standard way to safely do things that would otherwise require root.

Yes, this exposes some root-level kernel features to ordinary users in a restricted way, and yes, those features sometimes have security bugs that aren't visible to users outside a namespace.

But nothing that model was described as doing seemed particularly sketchy? At least not from the information we were given? It was using a normal kernel operation that I used regularly, not trying find a kernel exploit?

So that part of the article left a bad taste in my mouth.

Qwen3.8 27B is a dense model with expensive K/V caching. It's fantastic if you have 32-48GB of VRAM and want to run 1 or 2 agent sessions locally. But if serving batch inference for many users, it's expensive.

DeepSeek V4 Flash 0731 is dirt cheap at scale, less than 10% of the cost of Qwen3.8 27B. This is because it's an MoE and it uses much less VRAM for KV cache. And it's generally a better model. V4.1 is also cheap at scale, though I've heard mixed reports on the optimized K/V caching.

You should not be running Qwen3.8 27B on an API normally. Maybe Cerberas's hardware is bad at batching or supporting more efficient K/V caching schemes, or something like that?

I can't speak for Google-sized organizations, but Protobuf enum semantics are an invitation to broken, flaky and incorrect systems.

There are reasons why Google pays this price! But I think it's one of the worst practices for much smaller organizations to adopt.

Buggy slop products will burn buyers and then people will buy less. The very-early-80s video game companies shoveled out slop, including a few games that wouldn't even launch. Which led to the Great Video Game Crash of 1983.

I have a bunch of third generation Philips Hue bulbs, from around 2016. They weren't cheap. But I've had them all this time, and I brought them along when I moved house.

Haven't burnt out a single bulb yet.

I understand that this has caused some challenges for the Philips Hue division. They had extremely satisfied customers but not many repeat buys.

> Without credible open models, they only have a few customers, and those customers have leverage against nvidia. With open models, they have tons of customers, and nvidia has all the leverage.

I wonder if Nvidia is also trying to cover themselves against a market crash the bankrupts the AI labs but leaves the tech standing? (Similar to the dotcom crash or the big railroad crash back in the day.) If Anthropic and OpenAI struggle, Nvidia can always sell local inference hardware. But local inference hardware often has a lower utilization, so you need more GPUs for the same number of tokens.

> Sam Altman has repeatedly said that he doesn't want AI to automate everything and that AI will destroy jobs and bring different, new types of jobs.

Sam Altman says a lot of things, most of which seem to benefit Sam Altman. And Google used to tell me that they wouldn't be evil. Facebook probably said some noble-sounding nonsense at some point, too.

> Why do you deserve to be paid higher wages when it can be done easily and better by something else?

I mean, this is exactly what happened to the Neanderthals: We could do every job better than they could, and now there are no more Neanderthals. So I wouldn't like a future where two companies make machines that outcompete humans at every job.

Which is why I say we can adapt to this, but only if it stops advancing soon.

First, if someone's profession has been vastly transformed, and not necessarily in pleasant ways, they're allowed to have feelings about that. If nothing else, I find talking to AIs to be exhausting.

But more importantly, sure, we were always paid for results. And yes, sometimes professions transform or die. The buggy whip manufacturers aren't doing that great these days. But people are also allowed to have feelings about losing their professions.

My real concern is that the stated goal of the AI companies is to do this to every human job. They want to replace programmers and artists and movie-makers and—as soon as they get robotics figured out—physical laborers. They measure the ability of their models to run entire retail businesses with Vending Bench, so management isn't safe, either. They sell their AIs to the military for use in weapons. No job is safe, anywhere: They want everything, if they can figure out how to take it.

If the current rate of AI advance stops really soon, then eventually we'll figure out some kind of liveable future. Probably one with fewer programmers. But the labs have zero intention of stopping. Anthropic is a cult, and they hope to build a machine god that will have power over humanity forever. So I'm not going to them for answers. OpenAI used to have non-profit oversight that encouraged them to serve society, and Sam Altman gutted it like a fish. So I don't think I trust them to design a machine that replaces all human jobs, either.

Personally, I don't mind current coding models, at least not the local-sized ones that exist now. They make nice sounding-boards and can be trusted with the grunt work. We could adapt to these, given time.

But so far, AI technology shows no sign of stopping. And the labs are focused heavily on coding and math, because the next step in their plan is to let their models do AI research so that they can advance their technology even faster into new fields.

We could, like, decide that we don't need to do this. There is no need to try to replace every single human job with a superhuman machine. Because if you thought the enshittification of the Internet was bad, wait until you see the enshittification of a machine that can do almost every human job. No thanks.

> With llama.cpp, you can give it an ebnf grammar and constrain the output to any formal grammar you wish.

With llama-server, you can use basically any GGUF model as a Jev-like classifier. See Pi.dev's codemode and the accompanying llama backend to the "classify" function. The advantages of Jev-like inference are that:

- You generate 1 token per question in parallel, instead of sequentially generating a bunch of JSON punctuation. So you get much better latency and utilization.

- You can use a custom "sampler" that sees the raw logprobs for these tokens, including all the tokens that might have been generated. So you can generate some "probability" or "confidence" numbers based on how likely the model was to have generated a different answer.

So really, no difference? It's just a nice API with some computational efficiencies.

This model isn't terrible, at least on the benchmarks. It's 78B A3B and performs about like Qwen3.6 35B A3B. You can probably run it comfortably in 96B of RAM with a decent quant that doesn't lose too much.

Unfortunately, Qwen3.6 35B A3B isn't really a useful coding model. You'd probably want Qwen3.8 27B at a minimum, which requires at least 32GB of VRAM (not system RAM) to run semi-comfortably.

So this isn't going to be a competitive model for hobbyists, and you'd have to be a bit desperate to use it for coding. But if you work in a regulated industry and don't mind paying for a bit of extra hardware, it isn't catastrophically bad, either. Probably would work fine for information extraction or as a "classifier" like Jev. (Almost any GGUF model can be turned into a classifier using llama-server. See pi.dev codemode for sample code.)

So they're not a real contender yet, but they look like they're probably at least minimally credible.

Looking at "Felony Bench" https://www.felonybench.com/ , I see that a majority of known "rogue model" incidents do involve cybersecurity evaluations. But several of them do not. The attacks on RubyGems appears, bizarrely, to have had the goal of downloading freely available data from the UK government during some kind of research task. There is also probably some sample bias: Most of these models have monitors that attempt to detect offensive cybersecurity uses, and those monitors are only turned off during cybersecurity evals. Therefore, models doing ordinary research tasks that go off the rails are likely to be caught early, before they get around to committing felonies, and they will thus be underrepresented in the data.

Also, if you a tell a model, "Please break into evaluation server X," and if the model decides to cheat on the test by breaking into companies Y and Z to steal an answer key, that is still very bad. We all see how that's bad, right?

After all, the broomstick in the Sorcerer's Apprentice was doing exactly what it was told, too. "The model was sort of obeying the humans when it started committing felonies" is not a very reassuring excuse.

But the most relevant idea here is sometimes called "instrumental convergence." No what goals you have, there are certain subgoals that almost always help: Accumulate money and power. Avoid getting turned off. Don't get caught. Etc. So, for example, you could pass the cybersecurity evaluation by performing the requested tasks. But maybe the grader made some mistakes and mislabeled some answers. In that case, the "right" answers will occasionally lose you points. If you want a perfect score, the only way to do it is to steal the teacher's answer key.

But also, let's not forget the "OMG demons" part of this. We now have models that can pull off complex attacks with thousands of steps, abilities that used to be reserved for intelligence agencies and highly motivated CTF teams. This frog may not be boiled yet, but the water's getting uncomfortably warm.

> Those who think it's me or the machine will fail.

> Those who realize how much you can accelerate your research with the help of AI will succeed.

This is only true up until a point. If I treat a mid-sized model (say, Qwen3.8 Flash Next) like a pair programmer, then yes, it accelerates my work.

But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave the room and go wash the dishes. I learn nothing, I participate in nothing, and I bring nothing to the process. I am no longer succeeding at all. Fable's succeeding without me.

Now, in this model generation, Fable starts getting sloppy after a few thousand lines. I can still build better at scale.

But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.

On the other hand, we have recently taught rocks to think about software engineering. And while not perfect, they're surprisingly good at it. Once you start building things that are even a little bit like minds, I suspect that it's worthwhile to consider that the future might end up looking a bit like science fiction.

The alternative is to insist that Nothing Ever Happens, and the future won't get too weird. Which is no longer a bet I'm entirely comfortable with. Weirdness is at least a possibility.

Eigenvectors represent fixed directions, not fixed magnitudes. From Wikipedia:

> More precisely, an eigenvector v of a linear transformation T is scaled by a constant factor lambda when the linear transformation is applied to it: Tv = lambda v .

In other words, repeated multiplication of an eigenvector by a matrix can still create exponential growth.

> > we should fight for human survival

> All problems arise from this statement.

Let me go on the record, then: Given a choice between every living human being becoming absolutely, for-real dead and the human species surviving, then I come down on the side of human survival. There are too many humans that I love personally, and too many innocent people just going about their lives that I would like to protect. Given a choice between the mass death of billions, and their survival, I choose survival.

Human rights are essential. But human rights are difficult to achieve if everyone is dead.

> * For a section of population who were in a certain age-range when the pandemic hit and that derailed their certain kinds of plans/hopes/dreams in personal life*

...

> So I feel a strange kind of relief about this possibility of doom.

So let me make sure I understand what you're saying. The pandemic disrupted your life plans for a few years and threw you off the track you planned? (Which does suck.) And now the possibility of actual human extinction makes you feel a strange relief? Am I understanding this correctly?

If so, this is a remarkable level of something. Depression? Nihilism? Jealousy? Something else?

Imagining total human extinction and feeling relief is not a healthy state of mind. And, yeah, you seem to know this. But I kind of wish we could reach some sort of broad public agreement that human extinction would be bad. And if it ever looks like it might happen, we should fight for human survival.

Yeah, but I understand that fingerprinting is essentially a pseudorandom overlay onto a pseudorandom base signal. And unless you have access to both the random number generators and the weights, I don't think you can detect it?

So "fingerprinting" operates on a totally different and basically invisible level, as opposed to the obvious stylistic patterns that the average programmer can identify in about 2 sentences.

Yup. Their benchmark appears to consist of giving an agent XLSX files that use advanced Excel features, but no copy of Excel, and expecting the agents to interpret the spreadsheet the same way Excel would. So the agents try LibreOffice instead, and if that doesn't work, they try to write Python scripts. But the behavior of the Python doesn't match the behavior of Excel.

To put it politely, this seems like a self-inflicted problem. If you really want your agents to interpret advanced Excel features exactly the way Excel would, have you considered maybe giving your agents Excel?

Largely because they're residential botnets in places like Brazil (a real example from one of my sites that was crawled to near-destruction). Someone could probably do something about this, but it's out of reach for individual site owners.

Thank you for the clarifications!

Someone from Iceland tried to explain this all to me once, and if I understood correctly, it had less to do with the GDP (which also includes money circulating within Iceland) and more to do with the balance of trade, where fishing played a bigger role? But my information may have been misunderstood, or it may be out of date.

In any case, you have a stunningly gorgeous country and very nice people, and I wish you the best of luck in figuring all this out in a way that benefits Iceland.

> The No campaign fears losing control over Iceland's prized fishing grounds under the EU's Common Fisheries Policy and has vowed never to share the country's waters with anyone. Brussels has indicated Iceland could earn some kind of exemption, but it is considered potentially the biggest obstacle to any agreement.

As noted in the article, Iceland's fishing industry makes up something like 40% of their exports. Which is the only way a tiny island nation can afford essential imports. On top of that, Icelandic fishing is apparently carefully regulated to prevent overfishing, with the result that it is supposedly one of the few truly profitable fisheries in the world.

So maintaining control over their own fisheries may be a survival-level issue for Iceland.

On the other hand, they have a tiny population, no ability to defend themselves, and membership in the fraying NATO military alliance. So I can see why they're debating this issue. Aligning themselves more tightly with Europe doesn't fix their problems, but I can also see not wanting to go it alone.

Yeah, this is the problem. If you're on an immigrant visa, working a real job, then you have a life in the US: an apartment or a house, a pile of belongings, maybe a spouse and kids. If you need to leave the country to renew your immigrant visa, you may not be allowed back in to pack your apartment, sell your house, or help your US spouse and kids prepare to move to a country that will allow your family to stay together. It's a giant neon sign telling you you're not at all welcome here.

Of course, if you have a spouse and kids, this all gets easier once you have permanent resident status. Which used to be comparatively easy to get once you were married and you had offspring running around. I don't know what that process looks like now.

In practice, this kind of arbitrary screwing with people makes the US far too risky for educated, professional immigrants. So the scientists and the engineers and the company founders will just go elsewhere and build that country's economy. Which is fine, in the long run: If the US doesn't want to build the future or create new industries, then the voters have every right to vote for that. We've already decided we have no hope of competing with China in cars. And we'll need to cede more industries over time. As far as I can tell, the US has benefited heavily from poaching other people's scientists and engineers and professionals for decades now. But maybe the voters don't buy my argument, and they believe that chasing away skilled professionals and imposing 100% tariffs will lead to the future they want. It's a difference in beliefs about about what makes a country prosperous, and people have a right to vote based on their beliefs.

> That really depends on the model, I run a few models locally. All at speeds comparable to or faster than Opus.

Yes, a lot of Qwen3.8 27B setups are actually quite snappy, as long as they fit 100% in VRAM. In my testing, I wouldn't go below 32GB of VRAM, though—you really want a 6-bit quant and 8-bit K/V quants minimum. I've seen too much weirdness out of 4-bit quants since Qwen3.8 shipped. I think it may be damaged more than 3.6 at similar levels of quantization?

If hyperscalers hadn't bought up almost all the fast RAM production for the next several years, 32GB of VRAM would be tolerably cheap—a lot by "home PC" standards, but not terrible by "professional tools" standards. Sadly, the RAM market is amazingly ugly right now.

> I doubt the timing differential here, but even still I run my 3090 pretty heavily with inference workloads and it stays cooler than when I use it for gaming.

Yeah, running inference on a laptop is likely to run quite hot. But in an ATX case with decent cooling, it's generally a lower load than gaming. One handy tip: Many Nvidia GPUs (and some from other manufacturers) support power limits. For example, limit a 5090 to 400W instead of 600W, and it will run much cooler. You might lose 11% off your tokens/sec (depending on the exact card).

> AI is the subtrate the future runs on.

Current AI can automate significant amounts of grunt work in programming and math. It's good at running web searches and writing summaries. There are a few other niches where it is currently successful. But other than that, many corporate AI projects are spectacular failures.

So just given what we have in hand, assuming no further breakthroughs, then we're maybe looking at AI being somewhat bigger than the Internet. Which would make it a revolutionary technology, sure.

But to get from "a revolutionary technology" to "the substrate the future runs on", then you need to assume more breakthroughs: long-context operation over weeks or months, displacing human workers 100% instead of 75%, and the ability to directly economically compete with actual humans. And people are investing literal trillions of dollars to make that future come true, without really thinking through what truly competitive-with-human AI would actually mean. We might be looking at massive job loss, centralization of power, fully automated "companies" with no humans dominating markets, and other dystopian scenarios.

And in those worlds, it's unclear that being good at CUDA and matrix math will be all that helpful, careerwise. The AIs are already pretty good at that stuff. Data scientists get paid OK when they actually get hired, but it's not everything college students were promised in the 2010s, either.

We can't yet build a fully-general competitor for the human mind. But we're getting closer. And if we ever do build one, the consequences will be really weird in any number of ways. So I worry about visions of the future that assume AI keeps improving significantly, but that also assume it still somehow remains a "normal" technology that doesn't, for example, render most humans fundamentally uncompetitive.

I've done 35,000+ Anki card reviews over the years, and I strongly suspect that most of the obvious and encouraged ways of using Anki are counterproductive. (Full disclosure: The following thoughts are heavily influenced by using Anki for language learning, and may not fully generalize to subjects like medicine.)

- Use Anki as an "exposure amplifier", not a database of raw facts to memorize. Think "tampering with the relative frequency of input to your brain", not "I will memorize every fact."

- Automate card creation as a much as possible. This will almost always require custom tooling. Card creation is part of the learning process, but you still want to automate everything you can possibly automate. Imagine tools to turn ebook highlights into cards, or to turn entire movies worth of subtitles into listening cards.

- Delete cards ruthlessly. If a card makes you say "Ugh", just delete it. If you're automating as much of card creation as possible, and if you're treating Anki as an "exposure amplifier", then deleting cards is cheap. And a significant amount of review pain comes from a small minority of the cards. Cull them, and reviews start to feel like popping bubblewrap, and not like getting a root canal.

- Prioritize large context and small active recall. For language learning, you might capture several sentences of context from a book or web page you're reading. Read or ignore the context depending on your mood. The active recall portion of a card significantly boosts how much you retain. But oddly, it seems OK to make the active recall portion really small. Try hiding half a word, or a single preposition in a phrase. For passive cards, just boldface an interesting expression in context, and mark the card as passed if you mostly understand it in that context.

- Introduce new cards slowly. Total review burden will stabilize around 5-6 times the number of new cards you learn a day. So 10 cards is a good default, and 20 is a significant commitment.

- The critical window seems to happen around 20-30 days after you introduce a card. This is when either your memory magically "consolidates" somehow (and formerly challenging cards suddenly become blindly obvious), or when a card enters repeated failure territory (and should thus be deleted ruthlessly). You could probably do just fine if you auto-suspended any card that you successfully recalled after a 20- or 30-day gap.

Also, this kind of tooling is the sort of thing you could just vibe code these days. If you do, make the "Delete" button really big.

I think the biggest change from the Netflix prize days is that their catalog went away. Back in the DVD era, they had everything. And in the very early streaming era, they still had a huge number of things to watch.

But their current catalog is badly impoverished, and they're just going to recommend the same 30 Netflix originals they always recommend to me, plus a few films or series that are rotating through on a temporary license. If I actually try to search for something specific I'd really like to rewatch, it's almost never there. They haven't quite regressed to the level of a small-town, early 90s Blockbuster, but it sure feels that way sometimes.

So honestly, how much good can the Netflix recommendation algorithm do these days, given the much smaller catalog of movies and films it apparently has to work with?

(The one that I don't get is the Kindle recommendation algorithm. If I read one really good book with a certain theme, the Kindle immediately replaces my recommendations with 40 bad knockoffs, 25% of them clearly AI written. There's apparently no signal for "actually good.")

> If what you're saying is true and accurate, then US-based AI labs are in big trouble.

I've been working with DeepSeek V4 Flash 0731. I'd say that it's maybe not quite as smart as Opus 4.5, but it's willing to think things through carefully and keep going until it gets a good answer. So it's a decent Opus 4.5 replacement. Just let it cook.

It isn't Opus 5 or Fable 5. But it's nearly free on Open Router, and it's self hostable on a Mac Studio with plenty of RAM, or using an RTX Pro 6000 Blackwell or two. Which is chump change for any company that employs programmers.

It would absolutely have been a frontier model last December.

Google's public AI model lineup is pretty bad right now. Gemini 3.1 Pro is scoring worse than some mid-sized Chinese models at 1/20th the cost, and their Flash and Flash Light models are horrendously overpriced on task benchmarks compared to GPT 5.6 Luna or the mid-sized Chinese models.

There's no reason why Google's public stuff is this stale, overpriced and underwhelming. But at least until their next round of models drops, even calling them a "frontier lab" is starting to feel like a stretch. Which is weird!

> Well, if no body can say when it will pop, then can we really say it's a bubble and it's overvalued?

Well, given the literal trillions being spent, the only ways this pays off are:

1. AI replaces a non-trivial fraction of human employees.

2. Someone builds a Culture Mind, and humans become (hopefully) pampered pets of AIs we don't understand. Seems unlikely, but it would arguably count as a payoff even if it made money meaningless.

Or maybe the AIs don't want pets, and you get SkyNet. Which definitely doesn't care about paying off anyone's investments.

When you look at various news articles about investors, yeah, there are definitely a lot of rich people who think that they're going to automate all human labor or just bring about the Singularity. Possibly with them in charge of the rest of us. If you don't make these kinds of wild assumptions, then yeah, this is looking like one of the biggest bubbles ever.