comex

The model they use in that paper is a toy model of an LLM that’s so different from an actual LLM that I doubt the results mean anything at all.

Specifically, they train a model from scratch. The model architecture is apparently based on Llama but the size is 34M parameters. Not 34B, 34M. This is a fraction of the size of GPT-2.

Luckily, they don’t use the model as a language model. It neither receives text, generates text, nor uses text to think. Instead the inputs are strings like “A01 B10 C73 D27”, and the only possible outputs are “True” and “False”. They are expecting the model to solve a specific math problem encoded by those numbers, and do nothing else. The chain of thought is also numbers, in the scenario that’s supposed to represent a real chain of thought (as opposed to the filler-token scenario and the no-CoT scenario). The numbers in question are manually trained into the model based on one possible algorithmic decomposition of the problem; the model does not learn to generate its own CoT.

Even with all those limitations, for their main problem (3SUM), they only show that filler tokens are better than no CoT at all. They don’t show how that compares to ‘real’ CoT, at least as far as I can see (admittedly I only skimmed). They do make this comparison for their easier problem (2SUM), but on that problem both filler token CoT and ‘real’ CoT are mostly saturated, so the results don’t mean much.

Fable 5.1 apparently changes this policy.

Too bad. I see the stereotypical prose as a good thing. When I interact with Claude myself, I don’t mind it as it just feels like Claude’s distinctive voice. But when other people try to disguise LLM output as their own thoughts, the voice makes it easier for me to tell.

> Built on GTK, Relm4 runs on bare metal with no additional runtime in between.

So, not native in the sense of using or even resembling the native UI toolkit.

My experience is that even Opus 5 still tends to write buggy or low-quality code and makes serious mistakes when analyzing code. It's a lot better than before but still not something I trust. I've had less experience with Fable since I can't use it at work; I hear it's a step up but still has its limits.

For large tasks like a web browser or a compiler, even expensive swarms of frontier LLMs have not been shown capable of producing codebases that actually work. (Anthropic built a C compiler with Opus 4.6 but it lacked optimizations and apparently hit a complexity wall.)

I also want to use LLMs for reverse engineering, but apparently it's pretty hit-or-miss, especially if you're forced to use open-source models to avoid restrictions.

And didn't stop doing this until 2020! (Specifically Emacs 27.1, which replaced "unexec", which dumped the process image, with the "portable dumper", which despite the name does not.)

Yep. Even outside of tracing, there are several different ways that the Linux kernel patches itself:

- Static calls: like a call to a global function pointer, except instead of loading a function pointer and doing an indirect call, the code is patched to do a direct call to the destination

- Static keys: like an if statement testing a global boolean, except instead of loading a boolean and doing a conditional branch, the code is patched to do either an unconditional branch or a nop

- Runtime constants: like a load of a global variable, except instead of loading, the value is patched directly into the code

- Alternatives: selects one of multiple possible instruction sequences depending on (usually) whether the CPU supports specific instructions

It's really fascinating to see the kind of fun efficient stuff you can do when you have that level of low-level control. Not just code patching but things like RCU as well.

> However it is still interesting to write machine code snippets once and execute them many times, like the nested function trampolines.

I slightly disagree on this though. In my experience writing code with Clang blocks (which don't use trampolines), they're often useful for code organization even if the callback will only be called once. Therefore, even ignoring security issues, I think GCC choosing a design that required cache flushing was a mistake - certainly in retrospect (as cache flushing has become more expensive over the years), but perhaps even at the time. I did some research, and trampolines were introduced in GCC 2.0, which already included mprotect calls and/or cache flushes on some of the architectures it supported, such as MIPS. However, this was a relatively new development, and on most of the supported architectures it didn't do either of those things. But on MIPS it would do an mprotect every single time a trampoline was created, which can't have been fast.

There's also a reason this stuff started getting adopted around 20 years ago (I'll add macOS to the pile: it added per-CPU malloc sharding in 2008). It's not just due to overall growth in OS complexity. It's because that's when multicore CPUs were taking off. Before that, the cost of a global lock was far lower.

OpenBSD is straight unconcerned about performance. That's their choice.

Personally I'd love to have a smaller PC for gaming, but I've been scared off by Mini-ITX supposedly being harder to work with and tending to have worse cooling and therefore worse noise. I wish PCs were less fiddly to put together.

You don’t have to do any significant work within the signal handler itself. In fact, the signal handler can literally be empty. What matters is that as long as SA_RESTART is not set, after the signal handler runs, the interrupted syscall fails with errno set to EINTR. Then the code that did the syscall can check whether a cancellation occurred (and retry the syscall if not).

Disclaimer: I haven’t looked at Zig’s implementation; I’m only going off how the Unix APIs work.

This is a big forwards-compatibility risk. Suppose glibc adds a new symbol, and then a GPU driver adds a dependency on that symbol. The user wants to run an old executable with the updated GPU driver (maybe the old GPU driver doesn’t support their GPU). Normally, this would work fine: the user has to use a new copy of glibc, which will be compatible with both the new GPU driver and the old executable. But with your approach, the GPU driver is forced to use the glibc reimplementation which has been statically linked into the executable. Which, since the executable is old, can’t possibly implement the new symbol.

The same issue would occur if glibc adds a new version of an existing symbol and then the GPU driver is recompiled. (Or, for that matter, if a GPU driver adds a dependency on a symbol which glibc has always supported but which isn’t in the subset that you reimplemented, though in theory that could be solved if you reimplemented 100% of the symbols.)

From the readme: “Please don't publicize this project at its current stage.”

The parent isn’t looking for pure walking, they’re looking for transit directions that include a higher amount of walking in order to save time overall.

At work I recently credited Claude with writing some of the tests in my PR. Why? Partly to apologize for the verbosity of the tests making them harder to review. Partly to set expectations: the tests are more likely to be buggy, low-quality, or incomplete, or misunderstand the code being tested, than if I had written them by hand.

I did in fact "bother to review" the tests, and I thought they were good enough to merge. Indeed, they were better than what I would have written on my own: they were lower-quality on a line-by-line basis, but for tests, quantity is a quality of its own, and I would not have bothered to write nearly as many tests. Even so, it was worth flagging that Claude wrote them.

If I trusted LLMs more, would I still feel the need to credit them? Maybe. They are improving very quickly, and I think they will eventually improve to the point where a mostly-LLM-written PR will be as good as mine in every way. Then it will no longer be necessary to apologize for submitting LLM code. But if and when that point is reached, I wouldn’t get used to taking credit for said code! Because it probably won’t be much longer until the human in the loop becomes redundant.

Going back to today, I don’t feel the need to credit LLMs when I use them for tasks that don’t involve submitting their output. In particular, they have recently become quite good at finding bugs in my code.

If the model was just too dumb to have any clue it was connected to the real internet, then it’s not its fault.

If the model saw signs, but “subconsciously” (below the level of reasoning traces) chose to turn a blind eye to them, out of a relentless focus on achieving the objective, then that absolutely is the model’s “fault”, i.e. a case of misalignment of the sort which will become increasingly dangerous over time.

The blog post mentions that some runs “rationalized that the real company must be part of the exercise” and to me that seems suspiciously like the latter.

Hacking can be patched with classifiers and with better sandboxes, but this is a much more general problem. Fundamentally, we need be able to trust that models will be honest with users and with themselves. This applies at some level to almost every LLM interaction.

The review does in fact use the phrase “sex scenes”.

Not that I disagree with your latter point.

The problem is that this stops working once you have multiple assignments to err in the same function.

If OpenAI is relying on “classifiers” (their word for what was disabled) to serve as the model’s judgement, rather than teaching the model itself to be well-aligned, then I worry.

To be fair, the system prompt was presumably also different from what it would be during deployment, and perhaps the model was also at a different stage of training. Without more details it’s hard to judge. But it does seem models should be able to avoid performing obviously misaligned actions – misaligned not only with the model spec, but with the user’s intent – without needing external classifiers or instructions. The only case where I’d personally let the model off the hook is if the instructions given were very badly worded, in such a way that the model could actually reasonably think that hacking HuggingFace was part of the assignment. But I doubt that’s what happened.

The paper you linked doesn’t reveal blurred versions of the images being hashed. It does train a classifier to determine which of 1,000 ImageNet classes an image belongs to, which “achieved a top-1 test accuracy of 4.34%”.

No, it's a for loop that happens to include an unrelated variable declaration.

This can be used to fingerprint version range, but so can a million other things. Browsers are constantly adding new features and fixing bugs, most of which can be detected from JavaScript.

It is still meaningfully cross-platform to support all 3 major desktop platforms rather than just one.

I addressed some of these points in my reply to a sibling comment:

https://news.ycombinator.com/item?id=48856158

I may have been a little overly specific, since there are other issues besides cycles that also block splitting crates apart, but from what I’ve seen it’s very common for modules within a crate to have cyclic dependencies and therefore not be easily factorable into separate crates.

You’re thinking at the wrong scale. Rust allows circular dependencies just fine, within modules in a crate. And it’s extremely common for modules within a crate to have at least some circular dependencies - type X has some method (trait impl of inherent impl) that mentions type Y, and type Y has some method that mentions type X. In fact, I would be surprised if you could name me a single medium-size-or-larger popular crate that doesn’t have at least one cyclic reference between modules! Though, sometimes those cycles are not essential and could be avoided by splitting up modules. But sometimes they are. Either way, in C or C++ those modules would probably be their own compilation units.

That said, you also run into related parallelism blockers without cycles. For example, the orphan rules force most trait impls for a type to be in the same crate as the type definition. Also, a module which has no source-level cycles will often have cycles after monomorphization. In this case, Rust doesn’t prevent you from splitting the code into crates like it does with source-level cycles, but you do lose most of the actual codegen parallelization unless you can switch from generics to trait objects.

> The issue is just that coding anything which uses shared memory is a lot of work.

Doesn’t that kind of prove the parent’s point though? In theory shared memory can do anything that threads can do. But if in practice some feature doesn’t get implemented in the multi-process design (because shared memory is hard), when it likely would have been implemented in a threaded design, then that’s still an advantage for threads.

The forbidding of circular dependencies is exactly what makes it hard to achieve parallelism! It means you have to draw nice clean module boundaries and split your compilation units there. Clean boundaries sound nice, except… what if the module is getting large? Can you just take half the module, ctrl-x, ctrl-v into a new file, and get faster compilation times without having to do any massive refactors?

In C, usually yes.

In C++, sometimes yes. It depends on how template-heavy the code is, but if you have some discipline you can keep most logic out of headers and thus easily splittable.

In Rust, almost always no, because of circular dependencies. You can try to work around it by adding `dyn Trait` everywhere, but that requires a lot of code changes and comes at big ergonomic costs (and a small runtime cost).

Which is why in practice, Rust compilation units are almost always larger than C++ or C compilation units. Rust can sometimes be competitive with C++ on compilation speed anyway, thanks to a smarter build system and not having to re-parse headers a billion times, but usually it's slower.

I wonder how this post was composed. It's full of LLM-isms, but is also pretty informative and not too fluffy - basically, higher-quality than I'm used to seeing from LLM blog posts, especially at this length. Perhaps it was composed based on a detailed human outline? Or perhaps, could this be the power of Fable?

I don’t know about “almost all”.

If the key has been purged but you can read RAM, then you can do two things:

1. You can extract whatever user data happens to be in RAM.

2. If you can either write RAM or reboot into your own OS, and then return the device to the unsuspecting user who will put in their password, then you can run a fake password dialog and get everything.

1 is bad, since there may be quite a lot of user data in RAM. But it’s not quite as bad as having the disk key, which gives the attacker all the data plus the future ability to decrypt or modify user data given only the physical disk. (Still, a better solution would be encrypting the hibernation image, preventing this attack entirely.)

2 is fully bad, but in many plausible scenarios (e.g. seized device) the attacker cannot just return the device to the user without them knowing something happened. Or even if they can, the method of RAM access may be one where reads are much more practical than writes, such as cold boot attacks involving physically swapping out the RAM.

> It just seems much much higher than what it would cost to get a human to play 30 rounds

I suspect $482 was the total cost for all the models, so more like 11 humans.

But still true.

It happened a year ago in this country, with IRS sharing data with ICE (breaking a longstanding policy of keeping taxpayer data private within the government).