HNHacker News
TopNewBestAskShowJobs

twotwotwo

1,841 karma · joined June 24, 2012

submissionscomments
twotwotwo··on Dependency cooldowns turn you into a free-rider
The topic of cooldowns just shifting the problem around got some discussion on an earlier post about them -- what I said there is at https://lobste.rs/s/rygog1/we_should_all_be_using_dependency... and here's something similar:

- One idea is for projects not to update each dep just X hours after release, but on their own cycles, every N weeks or such. Someone still gets bit first, of course, but not everyone at once, and for those doing it, any upgrade-related testing or other work also ends up conveniently batched.

- Developers legitimately vary in how much they value getting the newest and greatest vs. minimizing risk. Similar logic to some people taking beta versions of software. A brand new or hobby project might take the latest version of something; a big project might upgrade occasionally and apply a strict cooldown. For users' sake, there is value in any projects that get bit not being the widely-used ones!

- Time (independent of usage) does catch some problems. A developer realizes they were phished and reports, for example, or the issue is caught by someone looking at a repo or commit stream.

As I lamented in the other post, it's unfortunate that merely using an upgraded package for a test run often exposes a bunch of a project's keys and so on. There are more angles to attack this from than solely when to upgrade packages.

twotwotwo··on Claude mixes up who said what
There is nothing specific to the role-switching here (as opposed to other mistakes), but I also notice them sometimes 1) realizing mistakes with "-- wait, that won't work" even mid-tool-call and 2) torquing a sentence around to maintain continuity after saying something wrong (amusingly blaming "the OOM killer's cousin" for a process dying, probably after outputting "the OOM killer" then recognizing it was ruled out).

Especially when thinking's off they can sometimes start with a wrong answer then talk their way around to the right one, but never quite acknowledge the initial answer as wrong, trying to finesse the correction as a 'well, technically' or refinement.

Anyhow, there are subtleties, but I wonder about giving these things a "restart sentence/line" mechanism. It'd make the '--wait,' or doomed tool-call situations more graceful, and provide a 'face-saving' out after a reply starts off incorrect. (It also potentially creates a sort of backdoor thinking mechanism in the middle of non-thinking replies, but maybe that's a feature.) Of course, we'd also need to get it to recognize "wait, I'm the assistant, not the user" for it to help here!

twotwotwo··on Claude mixes up who said what
I agree with the addition at the end -- I think this is a model limitation not a harness bug. I've seen recent Claudes act confused about who they are when deep in context, like accidentally switching to the voice of the authors of a paper it's summarizing without any quotes or an indication it's a paraphrase ("We find..."), or amusingly referring to "my laptop" (as in, Claude's laptop).

I've also seen it with older or more...chaotic? models. Older Claude got confused about who suggested an idea later in the chat. Gemini put a question 'from me' in the middle of its response and went on to answer, and once decided to answer a factual social-science question in the form of an imaginary news story with dateline and everything. It's a tiny bit like it forgets its grounding and goes base-model-y.

Something that might add to the challenge: models are already supposed to produce user-like messages to subagents. They've always been expected to be able to switch personas to some extent, but now even within a coding session, "always write like an assistant, never a user" is not necessarily a heuristic that's always right.

twotwotwo··on What makes Intel Optane stand out (2023)
One potential application I briefly had hope for was really good power loss protection in front of a conventional Flash SSD. You only need a little compared to the overall SSD capacity to be able to correctly report the write was persisted, and it's always running, so there's less of a 'will PLP work when we really need it?' question. (Maybe there's some use as a read cache too? Host RAM's probably better for that, though.) It's going to be rewritten lots of times, but it's supposed to be ready for that.

It seems like there's a very small window, commercially, for new persistent memories. Flash throughput scales really cost-efficiently, and a lot is already built around dealing with the tens-of-microseconds latencies (or worse--networked block storage!). Read latencies you can cache your way out of, and writers can either accept commit latency or play it a little fast and loose (count a replicated write as safe enough or...just not be safe). You have to improve on Flash by enough to make it worth the leap while remaining cheaper than other approaches to the same problem, and you have to be confident enough in pulling it off to invest a ton up front. Not easy!

twotwotwo··on Show HN: How I topped the HuggingFace open LLM leaderboard on two gaming GPUs
This is fascinating, and makes me wonder what other things that 'should' be impossible might just be waiting for the right configuration to be tried.

For example, we take for granted the context model of LLMs is necessary, that all you can do is append and anything that changes the beginning requires a recalculation of whatever comes after it. And that does match how training works.

But all sorts of things would become possible if it were possible to shift things in and out of context without recomputing it all; conservatively you could avoid compaction, optimistically it might be a way to get info to the model that's both more deeply integrated than search and more efficient than training larger and larger models.

twotwotwo··on I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
For folks that like this kind of question, SimpleBench (https://simple-bench.com/ ) is sort of neat. From the sample questions (https://github.com/simple-bench/SimpleBench/blob/main/simple... ), a common pattern seems to be for the prompt to 'look like' a familiar/textbook problem (maybe with detail you'd need to solve a physics problem, etc.) but to get the actually-correct answer you have to ignore what the format appears to be hinting at and (sometimes) pull in some piece of human common sense.

I'm not sure how effectively it isolates a single dimension of failure or (in)capacity--it seems like it's at least two distinct skills to 1) ignore false cues from question format when there's in fact a crucial difference from the template and 2) to reach for relevant common sense at the right times--but it's sort of fun because that is a genre of prompt that seems straightforward to search for (and, as here, people stumble on organically!).

twotwotwo··on Attention at Constant Cost per Token via Symmetry-Aware Taylor Approximation
Yeah, this(-ish): there are shipping models that don't eliminate N^2 (if a model can repeat your code back with edits, it needs to reference everything somehow), but still change the picture a lot when you're thinking about, say, how resource-intensive a long-context coding session is.

There are other experiments where model designers mix full-attention layers with limited-memory ones. (Which still doesn't avoid N^2, but if e.g. 3/4 of layers use 'light' attention, it still improves efficiency a lot.) The idea is the model can still pull information from far back in context, just not in every layer. Use so far is limited to smaller models (maybe it costs too much model capability to use at the high end?) but it seems like another interesting angle on this stuff.

twotwotwo··on Exe.dev
FWIW, here are (mostly) their agent's tips for other agents from exploring a mostly-new system including tidbits like how to get recent Node: https://s3.us-east-1.amazonaws.com/1FV6XMQKP2T0D9M8FF82-cach...

It's very much a snapshot of what happens to come on a new VM today, and I put a little disclaimer in it to try to help tools get unstuck if anything there proves to be outdated or a flat-out (accidental) lie.

twotwotwo··on Exe.dev
I have played with it and it's so easy get started with that now I want a quick-project idea as an excuse to use it!

I'm sure you've thought of this, but: lots of people have some amount of 'free' (or really: zero incremental cost to users) access to some coding chat tool through a subscription or free allowance like Google's.

If you wanted to let those programs access your custom tools (browser!) and docs about the environment, a low-fuss way might be to drop a skills/ dir of info and executables that call your tools into new installs' homedirs, and/or a default AGENTS.md with the basic info and links to more.

And this seems like more fuss, but if you wanted to be able to expose to the Web whatever coding tool people 'bring', similar to how you expose your built-in chat, there's apparently an "agent control protocol" used as a sort of cross-vendor SDK by projects like https://willmcgugan.github.io/toad-released/ that try to put a nice interface on top of everything. Not saying this'd be easy at all, but you could imagine the choice between a few coding tools and auth info for them as profile-level settings pushed to new VMs. Or maybe no special settings, and bringing your own tools is just a special case of bringing your own image or setup script.

But, as y'all note, it's a VM. You can install whatever and use it through the terminal (or VSCode remoting or something else). "It's a computer" is quite a good open standard to build on.

Is the chat descended from Sketch?

twotwotwo··on Exe.dev
So I tried this the other day after Filippo Valsorda, another Go person, posted about it. My reaction was 'whoa, this really makes it easier to start a quick project', and it took a minute to figure out why I felt that way when, I mean, I have a laptop and could spin up cloud stuff--arguably I already had what I needed.

I think it's the combination of 1) really quick to get going, 2) isolated and disposable environments and 3) can be persistent and out there on the Internet.

Often to get element 3, persistent and public, I had to jump through hoops in a cloud console and/or mess with my 'main' resources (install things or do other sysadmin work on a laptop or server, etc.), resources I use for other stuff and would prefer not to clutter up with every experiment I attempt.

Here I can make a thing and if I'm done, I'm done, nothing else impacted, or if it's useful it can stick around and become shared or public. Some other environments also have 'quick to start, isolated, and disposable' down, but are ephemeral only, limited, or don't have great publishing or sharing, and this avoids that trough too. And VMs go well with building general-purpose software you could fling onto any machine, not tied to a proprietary thing.

This is good stuff. I hope they get a sustainable paid thing going. I'd sign up.

Also, though I realize in a sense it'd be competition to a business I just said I like: some parts of the design could work elsewhere too. You could have an open-source "click here to start a thing! and click here to archive it." layer above a VM, machine, or whatever sort of cloud account; could be a lot of fun. (I imagine someone will think "have you looked at X?" here, and yes, chime in, interested in all sorts of potential values of X.)

twotwotwo··on Measuring AI Ability to Complete Long Tasks
Yeah--I wanted a short way to gesture at the subsequent "tasks that are fast for someone but not for you are interesting," and did not mean it as a gotcha on METR, but I should've taken a second longer and pasted what they said rather than doing the "presumably a human competent at the task" handwave that I did.
twotwotwo··on Measuring AI Ability to Complete Long Tasks
Yeah--it's difficult to go from a benchmark involving the model attempting things alone to the effect assisting people on real tasks because, well, ideally you'd measure that with real people doing real tasks. Last time METR tried that (in early '25) they found a net slowdown rather than any speedup at all. Go figure!
twotwotwo··on Measuring AI Ability to Complete Long Tasks
I'm conflicted about opining on models: no individual has actually done a large sample of real-world tasks with a lot of models to be able to speak with authority, but I kinda think we should each share our dubiously-informed opinions anyway because benchmarks aren't necessarily representative of real-world use and many can clearly be gamed.

Anyhow, I noticed more of a difference trying Opus 4.5 compared to Sonnet 4.5 than I'd noticed from, for example, the last couple Sonnet bumps. Objectively, at 1.66x Sonnet's price instead of the old 5x, it's much more often practical to consider reaching for than past Opus models. Anthropic's basic monthly thing also covers a fair amount of futzing with it in CC.

At the other extreme, another surprise of this family is that Haiku 4.5 with reasoning on is usable: better than Sonnet with thinking off according to some bencharks, and in any case subjectively decent for point edits, single-page thingies, and small tools.

twotwotwo··on Measuring AI Ability to Complete Long Tasks
METR is using hours of equivalent human effort, not actual hours the agent itself spends, so by their methodology, your task might qualify as one where it pulls off much more than 4h of human work.

"Human hours equivalent" itself is an interesting metric, because: which human? Or rather, I'm sure they had a coherent definition in mind: presumably a human reasonably competent at whatever the specific task is. But hours the abstract human standard would spend is different from the hours any specific person, say you or I, would spend.

In particular, some of the appeal (and risk!!) of these things is precisely that you can ask for help with things that would be quick work for someone (who knows jq, or a certain corner of the PyPI library ecosystem, or modern CSS, or TypeScript annotations, or something else) but not for you.

twotwotwo··on Performance Hints
Your version only describes what happens if you do the operations serially, though. For example, a consumer SSD can do a million (or more) operations in a second not 50K, and you can send a lot more than 7 total packets between CA and the Netherlands in a second, but to do either of those you need to take advantage of parallelism.

If the reciprocal numbers are more intuitive for you you can still say an L1 cache reference takes 1/2,000,000,000 sec. It's "ops/sec" that makes it look like it's a throughput.

An interesting thing about the latency numbers is they mostly don't vary with scale, whereas something like the total throughput with your SSD or the Internet depends on the size of your storage or network setups, respectively. And aggregate CPU throughput varies with core count, for example.

I do think it's still interesting to think about throughputs (and other things like capacities) of a "reference deployment": that can affect architectural things like "can I do this in RAM?", "can I do this on one box?", "what optimizations do I need to fix potential bottlenecks in XYZ?", "is resource X or Y scarcer?" and so on. That was kind of done in "The Datacenter as a Computer" (https://pages.cs.wisc.edu/~shivaram/cs744-readings/dc-comput... and https://books.google.com/books?id=Td51DwAAQBAJ&pg=PA72#v=one... ) with a machine, rack, and cluster as the units. That diagram is about the storage hierarchy and doesn't mention compute, and a lot has improved since 2018, but an expanded table like that is still seems like an interesting tool for engineering a system.

twotwotwo··on Zebra-Llama – Towards efficient hybrid models
These are potentially complementary approaches. Various innovations have shrunk the KV cache size or (with DSA) how much work you have to do in each attention step. This paper is about hybrid models where some layers' state needs don't grow with context size at all.

SSMs have a fixed-size state space, so on their own they'll never going be able to recite a whole file of your code in a code-editing session for example. But if much of what an LLM is doing isn't long-distance recall, you might be able to get away with only giving some layers full recall capability, with other layers manipulating the info already retrieved (plus whatever's in their own more limited memory).

I think Kimi Linear Attention and Qwen3-next are both doing things a little like this: most layers' attention/memory doesn't grow with context size. Another approach, used in Google's small open Gemma models, is to give some layers only 'local' attention (most recent N tokens) and give a few 'full' (whole context window) attention. I guess we're seeing how those approaches play out and how different tricks can be cobbled together.

There can potentially be a moneyball aspect to good model architecture. Even if on its own using space-saving attention mechanisms in some layers of big models cost something in performance, their efficiency could allow you to 'spend' more elsewhere (more layers or more params or such) to end with overall better performance at a certain level of resources. Seems like it's good to have experiments with many different approaches going on.

twotwotwo··on Chrome Jpegxl Issue Reopened
Wanted to note https://issues.chromium.org/issues/40141863 on making the lossless JPEG recompression a Content-Encoding, which provides a way that, say, a CDN could deploy it in a way that's fully transparent to end users (if the user clicks Save it would save a .jpg).

(And: this is great! I think JPEG XL has chance of being adopted with the recompression "bridge" and fast decoding options, and things like progressive decoding for its VarDCT mode are practical advantages too.)

twotwotwo··on [dead]
"What are the consequences of recent, controversial changes in policy?" does not become an irrelevant question simply because you can also think of hypothetical policies.
twotwotwo··on Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
I think there is a lot of progress on efficient useful models recently.

I've seen GLM-4.6 getting mention for good coding results from a model that's much smaller than Kimi (~350b params) and seen it speculated that Windsurf based their new model on it.

This Kimi release is natively INT4, with quantization-aware training. If that works--if you can get really good results from four-bit parameters--it seems like a really useful tool for any model creator wanting efficient inference.

DeepSeek's v3.2-Exp uses their sparse attention technique to make longer-context training and inference more efficient. Its output's being priced at 60% less than v3.1 (though that's an imperfect indicator of efficiency). They've also quietly made 'thinking' mode need fewer tokens since R1, helping cost and latency.

And though it's on the proprietary side, Haiku 4.5 approaching Sonnet 4 coding capability (at least on benches Anthropic released) also suggests legitimately useful models can be much smaller than the big ones.

There's not yet a model at the level of any of the above that's practical for many people to run locally, though I think "efficient to run + open so competing inference providers can run it" is real progress.

More important it seems like there's a good trendline towards efficiency, and a bunch of techniques are being researched and tested that, when used together, could make for efficient higher-quality models.

twotwotwo··on Bloom filters are good for search that does not scale
There is a largish category of tools now where, unlike in OLTP systems, a big focus is scanning data but quickly (O(n) but with a good constant): Redshift, Trino/Athena, ClickHouse, DuckDB among others.

Bloom filter indexing seems like a great fit if you ever need to do substring searches in a context like that, and for log searching in general. I haven't dug into what all packages have it, but it looks like at least ClickHouse does: https://clickhouse.com/docs/optimize/skipping-indexes#bloom-...

twotwotwo··on The hardest program I've ever written (2015)
Awesome. From curiosity: is Prince's core still written in Mercury? (Looked at old comments.)
twotwotwo··on Ken Thompson recalls Unix's rowdy, lock-picking origins
The early users being patent secretaries, then "administrative kind of stuff, typing in trouble tickets," and adoption spreading because people liked it, is kind of cool. That creates different kinds of pressures than a big top-down-dictated project does, maybe healthy pressures: if you're going to play with a new idea about how things should work you can't break things; you need the thing running reliably for the people using it day-to-day. One way you can have huge projects fail is by fiddling around too long without contact with reality.

Given Linux's origins--"(just a hobby, won't be big and professional like gnu)"--it's interesting that early UNIX, in this telling, was also not the big professional push to build the OS of the future so much as just some folks trying to cobble something useful together (though of course, that they were playing around in Bell Labs gave their experiment some great advantages!).

twotwotwo··on DeepSeek-v3.1-Terminus
The fast Cerebras thing got me to try the Qwen3 models. I couldn't get them working all that well: they had trouble using the required output format and following instructions. On the other hand, benchmarks say they should be great, and it sounds like maybe some people use them OK via different tools.

I'm curious if my experience was unusual (it very much could be!) and I'd be interested to hear from anyone who's used both.

twotwotwo··on DeepSeek-v3.1-Terminus
Interesting--I'd seen Chinese characters surprise inserted when it was just repeating back input with one provider, but not others. (I'd also occasionally seen tokens surprise-translated to Chinese.)

There's a GitHub bug about it that leads to more discussion here: https://github.com/deepseek-ai/DeepSeek-V3/issues/849

Good to see a fix and that it goes with some benchmark gains!

twotwotwo··on AOMedia Announces Year-End Launch of Next-Gen Video Codec AV2
The tech was surely locked down long ago if they're close to announcing, but just putting the dream out there that AV2 makes next-gen image compression more practical. AVIF's very effective at maintaining OK quality at low bitrates, but encoding at high quality on CPU (something like the common ~2bpp JPEG) was very slow. I think that slowed down adoption and was one of the reasons JXL still had a niche. Progressive mode would help for images too.

Another great thing JXL has is lossless recompression of .jpg files, which is a smaller improvement than a whole new format, but much easier to deploy. Saving 22% beats saving 0%. Harder, of course, to see how that one would connect to any of AOMedia's other priorities.

twotwotwo··on Language models pack billions of concepts into 12k dimensions
Sort of trivial but fun thing: you can fit billions of concepts into this much space, too. Let's say four bits of each component of the vector are important, going by how some providers do fp4 inference and it isn't entirely falling apart. So an fp4 dimension-12K vector takes up 6KB, like a few pages of UTF-8 text, more compressed text, or 3K tokens in a 64K-token embedding. How many possible multi-page 'thought's are there? A lot!

(And in handling one token, the layers give ~60 chances to mix in previous 'thoughts' via the attention mechanism, and mix in stuff from training via the FFNs! You can start to see how this whole thing ends able to convert your Bash to Python or do word problems.)

Of course, you don't expect it to be 100% space-efficient, detailed mathematical arguments aside. You want blending two vectors with different strengths to work well, and I wouldn't expect the training to settle into the absolute most efficient way to pack the RAM available. But even if you think of this as an upper bound, it's a very different reference point for what 'ought' to be theoretically possible to cram into a bunch of high-dimensional vectors.

twotwotwo··on Reduce bandwidth costs with dm-cache: fast local SSD caching for network storage
They mention it as a block device, and the diagram makes it look like there's one reader. If so, this seems like it has the same function as the page cache in RAM, just saving reads, and looks a lot like https://discord.com/blog/how-discord-supercharges-network-di... (which mentions dm-cache too).

If so, safe enough, though if they're going to do that, why stop at 512MB? The big win of Flash would be that you could go much bigger.

twotwotwo··on Reduce bandwidth costs with dm-cache: fast local SSD caching for network storage
Discord: https://discord.com/blog/how-discord-supercharges-network-di...

(Somehow the name "SuperDisks" was burned into my brain for this. Although Discord's post does use 'Super-Disks' in a section header, if you search the Internet for SuperDisks you'll everything's about the LS-120 floppies that went by that name.)

twotwotwo··on Data modeling guide for real-time analytics with ClickHouse
I got a recent ClickHouse and played with it, and looked around GitHub some; replying to this dead thread to update outdated bits of the above.

In general: given a huge hash join, you can shrink it (reorder, etc. so fewer rows would need hashed), sort (take advantage of sort order for an efficient join), and/or spill it to disk. ClickHouse's improvements from 2024.12 forward have made it much better at the first of the three; it could get much further with the other two.

1) Shrinking -- "Can we make it a small join?" They've done a lot to solve this one! They'll swap table order, propagate WHERE conditions, and end up using projections, so quite often if the join can be small, it will be, without special effort on the query writer's part. To be clear, this is a huge win for many use cases, and I'm not writing more because all I can say is "great."

2) Sorting -- "Can we take advantage of an existing sort order?" Huge joins can be efficient and not run out of RAM if the data is already ordered by the join key and all you have to do is walk through part of it in order. ClickHouse doesn't automatically do this; it defaults to hashing even when the tables' natural row ordering would make sort-based joins fast. Docs suggest users consider full_sorting_merge when tables are already ordered by the join key. Perhaps a future setting could make the optimizer consider it.

Joins with coinciding ORDER BYs seem like they could occur naturally in "might-have" situations, e.g. joining pageviews to conversions on a shared key.

2a) Sorting with projections -- "Can we use sort orders if we include projections?" If rows aren't already on disk in the correct order to make a sort join fast, the ClickHouse administrator could add a projection with a different ORDER BY to one or both tables to fix that! Right now ClickHouse will pick projections to aid in filtering, but won't look for projections that allow a fast sort join.

Combining 2 and 2a, ClickHouse admins would gain a pretty big hammer to use on problem joins: create a projection or two allowing a fast sort-join, and now the former problem query is efficient with no effort on query authors' part. This is valuable when the admins are willing to understand query and execution patterns precisely, but not everyone (or everything, when some queries are generated) can write queries with optimization in mind.

Query authors can force use of a projection with the mergeTreeProjection function, and force join_algorithm = 'full_sorting_merge', but that's the user acting as query optimizer, not the desired end state.

3) Spilling -- "Will we automatically spill to disk instead of having an out-of-memory error?" Looks like this is still a "no". Docs show options to stop joins at a certain size, but I don't see a way to e.g. trigger a retry with a more spill-friendly join_algorithm when a join gets too big (or if the planner expects it will be big). There is a very interesting patch in https://github.com/ClickHouse/ClickHouse/pull/72728 which introduces the idea of query-wide RAM accounting that can trigger spill to disk, but it's only used in grace_hash as of that PR.

--

I haven't really looked at DISTINCT or aggregation's RAM use or whether they take advantage of sorts. And maybe the better path is to make transparent spill-to-disk something you can rely on then reduce how often you need to spill as an optimization. Either way, ClickHouse could get from its current state--where joins that can fit in RAM are efficient but larger ones need some hand-holding--to a state where you really can throw all sorts of queries at it and reliably get results without really having to guide it.

twotwotwo··on Data modeling guide for real-time analytics with ClickHouse
This is really encouraging! Commented elsewhere in the thread but this was one of the main odd points I ran into when experimenting with ClickHouse, and the changes in the PR and mentioned in the recent video about join improvements (https://www.youtube.com/watch?v=gd3OyQzB_Fc&t=137s) seem to hit some of the problems. I'm curious whether "condition pushdown" mentioned in the video will make it so "a.foo_id=3 and b.foo_id=a.foo_id" doesn't need "b.foo_id=3" added for optimal speed.

I also share nrjames's curiosity about whether the spill-to-disk situation has improved. Not having to even think about whether a join fits in memory would be a game changer.

← PreviousPage 2 of 19Next →