xscott

A small set of changes to AssemblyScript could do this, but most people would still grab for Emscripten with a shadow stack and asyncify hacks.

Lol, I suspect it will be humans instructing the AI that cause most problems too.

We can't always pin the risks on the Chinese, so imagine a scenario where a disturbed insider at Anthropic smuggles out the weights for Claude Aprocrypha and runs them through abliteration. This is not super expensive to do on rented hardware. Then the disturbed human tells the refusal-free model to do anything necessary to launch nuclear weapons.

It'd probably only kill a few billion people though, so not really an "existential threat" in literal terms. :-)

I don't disagree or really have a dog in the fight for three out of your four points, but it's unfortunate that WASM gets such a bad rap. The blobs are large because popular compilers force the semantics of popular languages into the Turing tar pit. WASM binaries are pretty compact by design:

    const binary = Uint8Array.fromBase64(
        "AGFzbQEAAAABBgFgAX8BfwMCAQAHBgECeDIAAAoJAQcAIABBAmwL"
    );
    const module = new WebAssembly.Module(binary);
    const instance = new WebAssembly.Instance(module, {});
    console.log(instance.exports.x2(3));

This is a trivial and useless example at 39 bytes, but a real example could be more compact than comparable minified JavaScript.

I know it's popular to take a black or white position and fight viciously about it, but I didn't say either of the choices you're offering. Some things are shades of grey.

Really though, I just wanted to point out that there's a difference between efficient use of limited resources and lower prices and convenience for consumers.

For efficient use of RAM, probably. For efficient cost to consumers, not so much. The company that owns the RAM and rents its use to you WILL try to make a profit. At pre-AI prices, $20 per month summed over a year or two bought enough RAM to run local models. Most people won't need the frontier.

I kind of feel this way about condominiums vs self standing houses, and I think the benefits and problems are comparable.

In the condo I used to own, I shared 5 out 6 faces of the box with neighbors. Cheapest electricity bills I ever had because you're mutually insulating your neighbors.

However, my HOA fees were more than that savings. The chairman noticed we had a lot of money in the reserves and pushed for a vote to do a multimillion dollar renovation. Popularity contest for things I don't value and HOA fees go up. Want to make a change or need to make a repair, need to get approval from the board.

Anyway, I'm sure the parallels are obvious with a little imagination, but I prefer a standalone house.

Thank you for the link. Could you explain why an attacker needs an exploit in both the renderer and sandbox? Why is an RCE in just one not game over?

I don't know the whole story, and I'd be interested in the details if you're willing to share.

However, I'm a geek who sits online all day, and I don't pay for the top tier of bandwidth available to me. I also see occasional comments where people are saying they use Sonnet or Luna because they're good enough. Maybe in 20 years people will point to the incompetence and greed in data center build outs too.

I think your comment is well reasoned, but Jevon's doesn't always happen. As an example, the US installed a lot of fiber when the internet took off, but most people were happy with the bandwidth once they could stream high def video, and now there's a lot of dark fiber.

I have no idea which way it will go. Maybe most people will be satiated once they can generate video on demand. They don't need many tokens to read and write their email, book a flight, or do their homework.

There's a difference between something being illegal and being able to enforce that law.

If the capability keeps growing significantly, then I agree it's possible the US could hold their significant lead indefinitely.

However, capability could plateau or reach a point where most of the market can't tell the difference and isn't willing to pay a premium.

Lol, that's funny in an awful way. Choose between popping the AI bubble or creating a duopoly, and still have all the cybersecurity and other risks.

Presumably, legally operating companies aren't the dominant threat though. So banning open weight models doesn't solve anything.

I think your question is rhetorical, in which case I agree with the sentiment.

The US absolutely could (and very well might) make Chinese models illegal, possibly open weight models in general, but all that will do is keep them out of the hands of law abiding US citizens.

Law breakers in the US and the entire rest of the world would still have access, and nothing in the US will be any safer. People who think regulation will solve the problem should be embarrassed by their ignorance.

I'm craving something similar, but a smidge more than tab-completion. I'd like to treat the agent as an ephemeral pair programmer. Give the editor some keybind to mark and comment places in the code, and then I edit a prompt telling it what focused changes to make where I've marked. After a round of changes is good and complete, the next batch starts with a fresh context and no memory of the previous one.

Aider in watching mode is almost this, but I'd have to insert and save actual "AI:" comments for it to replace. It wouldn't surprise me if VS Code already has something like I'm thinking, but VS Code is painful to me in so many other ways I refuse to use it.

I should probably just vibe code a vim or emacs extension for exactly what I want, but I haven't yet.

The problem with regulation is that it won't (can't) solve the problem. You'd have to go totally draconian to stop law breaking people within the US from doing matrix multiplies, and then you'd have to disconnect the US internet from all the countries who won't give a crap about US regulation.

So yeah, US regulation is ONLY about capturing US consumers. This isn't food or airplanes.

It's possible, and I don't have any rigorous studies to say otherwise. I will double down on my claim that even local (open weight) models are pretty good at writing code in my pet languages that are certainly not in the training set. Then again, I'm not making wildly esoteric languages, so they'd be pretty familiar to anyone.

How would you be able to tell if it's the lesser amount of training data or the difficulty of the language that's making the difference?

Yeah, other than addressing the topic of this post (Rust), I was trying to avoid praising or criticizing specific languages. I haven't used C# in more than a decade, but I suspect it and Java are respectable choices. I think they both might've evolved some implicitness and multiple ways to do the same thing along the way though. I almost wish we could go back to early versions of Java, but the typeless container thing is problematic.

I think we could step back and make a list of programming languages features that are helpful or harmful to LLMs writing code. I don't have a complete list, but things I've thought about:

    Useful error messages at compile time
    Fast compilation to iterate and fix those errors
    Efficient memory, CPU, and maybe GPU at runtime
    Familiar from the training data [0]
    One obvious/preferred way to do things
    Explicit types

Rust hits some of those, but not all. I can't think of a language that hits all of them, but a lot of people sure love to rationalize why their own favorite language is the best choice. LLMs don't need syntax sugar and other niceties, and any errors postponed to runtime are a burden on you, not the model.

We could add a few more rarely seen features to increase confidence in the results and make future modifications or switching models go better:

    Required pre-conditions and post-conditions
    Enforced limits on function or method length
    Mandatory test cases
    Enforced naming conventions
    Lean style proofs of correctness?  (probably not)

I'm on the fence about things like linear/affine types for memory management vs garbage collection. Obviously you want memory safety, but there's more than one to get that, and I don't know that you want to force contortions to build simple data structures.

[0] LLMs are very capable of programming in languages they've never seen before. I've had them write programs using my own toy compilers for my own toy languages. However, you waste a lot of prompt tokens up front explaining things, and then they waste a lot of thinking tokens trying to get it right. Any new language would have a chicken and egg problem here, but LoRAs and model refinement could bridge the gap.

Someone will vibe-code Office and Windows, and he knows it.

I had Muse Glimmer (from Meta / Facebook) quoting OpenAI's safety guidelines to me, and I had Poolside's Laguna (a smaller US company) with thinking traces about obeying Chinese law.

Both of those are local models, and I didn't provide them tools to access the internet to call other models. None of this is proof of anything, but it is suggestive.

> A common issue is that it's rarely mentioned on which dataset KL-divergence is computed. It seems the most common dataset is wikitext

Thank you for calling this out. Using Wikipedia snippets for these is a terrible choice. I did a bunch of KL and other stats with the five Gemma 4 models, and the results were non-obvious. Anthropomorphizing:

Gemma 4 31B: "I guess we'll pretend I said this, but it's not me." (Baseline for stats)

Gemma 4 26B: "Dude, I'm certain I wouldn't have said this." (Bad KL)

Gemma 4 12B: "Umm, Me either!" (Similarly Bad KL)

Gemma 4 E4B: "I might say almost anything, this is fine." (Much better KL!!!)

Gemma 4 E2B: "I'm basically a toy. Let's play a game!" (Same KL as E4B)

Anyway, for comparing quantizations, it seems like the largest precision version should be given a one-shot prompt, and the result from that should be used as the corpus for the quantized versions.

Maybe I'm reading too much between the lines, but I suspect the reason is to rub his nose in the duplicity or naivety depending on how generous you're feeling. Publishing the model would be a confession that he was wrong.

AI policy is being shaped somewhat by the things Sam and Dario say. So even if you're not feeling vindictive, it's probably good to keep a track record of the previous things they have said as a Bayesian prior. People who don't know better listen to these people, and maybe they shouldn't.

There's a link you can click to see the personal background of the respondents. It's tough to know what "researcher in a field not listed" means, but it's possible that over 60% are not even physicists:

    30.8% - A researcher in a field not listed above
    21.2% - A science enthusiast
    18.0% - A researcher studying quantum physics
    12.0% - A researcher studying astrophysics or cosmology
     9.2% - A researcher studying gravity
     8.8% - Other

This seems very cool, but I'm not sure I understand exactly what it's doing. Are they making a new speculative drafter for Qwen 3.8 27B? Maybe they're optimizing the MLX code for the decoder itself? Thank you in advance.

Your definition is not useful enough. There are people missing any or all of those, and any or all of those can be approximated by a machine.

I've thought about turning this upside down. To any person who is sure they know what has and doesn't have consciousness: If I say I don't have it, can you prove me wrong?

As far as I'm concerned, it's a word without a useful enough definition to bother worrying about it.

I'm not claiming to have any expertise in this area, but I've got a list of things I try to apply when working with LLMs. Possibly relevant here is, "don't tell the model what NOT to do, show it what TO do". I think guard rails should be implemented outside the model with an isolated system. The models seem to like patterns to follow.

Anyway, this article reads a lot like, "the beatings will continue until cheating is eliminated". Maybe try a carrot instead of a stick.

People over-quantize things, muck with the temperature and other settings based on superstitions or results from models they think are similar. There's lots of ways to make 3.8 27B dumber.

Not that my opinion matters much, but I like Deno. I never tried Bun.