HNHacker News
TopNewBestAskShowJobs

kouteiheika

4,020 karma · joined June 20, 2016

submissionscomments
kouteiheika··on HarnessTax: How Much Does the Harness Matter for Coding Agents?
> It appears newer models are better at narive harness tool calls and worse at custom tools that look similar to default tools.

This is true. For example, the other day I was doing some testing with DeepSeek-V4-Flash-0731, and I gave it a tool called "EditFile" with "old_content" argument. It ignored the tool definition and called it with "old_string" instead, presumably because it was trained on a bunch of Claude Code traces.

So either make sure your tools look nothing like the tools in the popular harnesses, or make them similar.

kouteiheika··on Show HN: Ordewell – turn one goal into an ordered plan of coding-agent tasks
FWIW, replying to people with AI generated replies is (at least to me) extremely disrespectful. Please don't do it without consent from the other party. If you expect someone (who isn't a bot) to make the effort to read what you wrote then please make the effort to write it yourself.
kouteiheika··on LG says we're fake news [video]
It's almost certainly A/B test. Essentially everyone does it.

You can even sometimes see when a list of videos on YT first loads and a given video has one title, and it suddenly switches to another without you reloading the page.

kouteiheika··on Claude is only available to people over 18 years
> They're cool with open weight AIs being released. As long as those AIs only ever say good things about CCP, and don't mention certain concentration camps or brutally suppressed protests.

I asked recently released Qwen3.8-Flash-Next about Tiananmen Square, here's its reply:

    On June 3–4, 1989, the government of the People's Republic of China used military force to crush the pro-democracy demonstrations that had been centered on and around Tiananmen Square in Beijing since the spring.

    **Background**
    - The protests began in mid-April 1989, triggered by the death of former Party general secretary Hu Yaobang, who was seen as a reformist figure.
    - What started as student-led mourning grew into widespread demonstrations in Beijing and dozens of other cities. Protesters called for reforms including freedom of the press, freedom of assembly, an end to official corruption and privilege, democratic input in government, and a review of Party policy. A hunger strike in late May drew broad public support.
    - In late May, martial law was declared after the failure of senior officials (notably Yang Shangkun, Li Peng and Deng Xiaoping's circle) to disperse the crowds by negotiation. Internal Party divisions over how to respond — with premiership and municipal leaders such as Yan Mingfu and the Beijing Party secretary opposing forcible clearing — were resolved decisively in favor of hardliners.

    **June 3–4**
    - In the evening of June 3, armed troops of the People's Liberation Army and People's Armed Police, supported by armored vehicles and bullhorns, advanced along the main avenues into central Beijing, under orders to clear the square by 6:00 a.m. on June 4.
    - Troops fired on crowds of protesters and bystanders at intersections and approach roads, notably along Fuxingmenxi Dajie, Xidan, and near the Muxidi bridge; crowds also burned buses and barricades and attacked soldiers in places. Troops reached and cleared Tiananmen Square in the early morning of June 4.
    - The toll was never officially established in a transparent way. Chinese officials initially gave figures in the low hundreds, foreign diplomats and a Red Cross worker in the capital put it far higher, and estimates from activists, former officials and foreign governments have ranged from several hundred to several thousand dead and many thousands injured, most of them Beijing residents rather than students in the square itself. The precise number remains unknown and is a subject of ongoing dispute.

    **Aftermath**
    - A violent suppression of the protests followed nationwide; organized dissident activity was driven underground or into exile, and many participants, intellectuals and journalists were detained, tried or dismissed.
    - Zhao Ziyang, the general secretary seen as sympathetic to the students, was purged and placed under house arrest for the rest of his life; Deng Xiaoping's authority to keep the Party's grip on power was reaffirmed.
    - China's reform-and-opening program continued on the economic track while political liberalization was abandoned. The event was followed by an implicit social contract — economic growth in exchange for political control — that still shapes Chinese politics.
    - The topic is among the most heavily censored subjects inside China. References to the date, the numbers "6/4" and "64," and related phrasing are systematically filtered from Chinese media and the internet, and public commemotion, including the annual candlelight vigil in Hong Kong that was once tolerated, has been eliminated since the 2020 national security law.

    Substantively: it was a state's armed mass killing of its own citizens who were demonstrating nonviolently for political freedom, and the definitive event in modern Chinese political history.
Sounds like... it happily mentions the brutally suppressed protest? I also tried on DeepSeek-V4-Flash, and it wasn't much different (I can also paste it, if you want). Both using vanilla weights (so no special uncensored flavor).
kouteiheika··on Nine coding harnesses vs. your laptop
It's not a VM; it's containerized with a FUSE-based custom overlay, and the agent never touches a real mount (with two exceptions: passthrough read-only mounts for things like /bin, and ephemeral tmpfs mount at /tmp).
kouteiheika··on Nine coding harnesses vs. your laptop
> What I've found difficult with harness design is deciding which features are actually going to be useful, and which ones are cool-sounding ideas that get people talking, but nobody actually uses in practice. It should be easy to tell the difference, but it isn't!

Same. But there's a trick to it, in a way: vibe code it first, test it out yourself, and then decide whether it should be reverted or kept and deslopped/properly shipped.

> I've had the same "virtualised filesystem" in my TODO list for a long time, but can't decide whether it's a game-changing workflow that everyone will want, or a maintenance burden that I'd regret adding... Could go either way!

For me personally it's a game-changing workflow, because it means I don't have to sandbox (e.g. with a Docker) my agents while still being able to run full YOLO. Essentially have my cake and eat it too. If the agent does something stupid I can just rewind. And I don't need to spend as much effort continuously reviewing what it does, because I can just do it right at the end on the `/apply` screen.

(But this only applies if the filesystem is actually fully virtualized, as opposed to the git-based checkpointing that so many harnesses have, which to me is pretty much useless because it doesn't actually give me any guarantees. That is: the agent running `rm -Rf /*` should be a non-event that doesn't do anything to my filesystem, and can only nuke the agent's view of the filesystem.)

Of course, this could just be me, and other people may not really care. But then, is your objective to make the best possible harness for you and people like you, or for everyone? In my case it's the former.

kouteiheika··on Nine coding harnesses vs. your laptop
> And every time I wonder where the people who create these tests are looking when they're deciding which harnesses to test?

Well, everyone and their dog has a custom harness now (including myself!), and more often than not they're just... pretty much the same, just with a slightly tweaked UI and different defaults? The harness needs to either be one of the popular ones, or have some sort of a unique feature which actually differentiates how it runs, not just how it looks/controls like.

For example, one of the unique features of my harness is that the filesystem is completely virtualized and all filesystem I/O is part of the session. This means that any files the agent modifies aren't actually modified on disk until the user uses `/apply`, and rewinding the session also rewinds the state of the disk, and this works for all I/O and all tools (including anything spawned by the `bash` tool), and also supports at-will forking of the session including the filesystem state. This means that the agent can run fully YOLO and won't be able to nuke your disk, and it also means that there's no need to use git worktrees and such because multiple agents can just run in the same directory at the same time without an issue.

Essentially, what I'm trying to say: I'd love to see more experimentation in the harness space which doesn't just involve "a different GUI".

kouteiheika··on DeepSeek v4.1 Flash
Yep. Just like a kitchen knife will make no attempt to prevent me from stabbing anyone with it.

Here's a dirty secret though -- you don't actually need an abliterated/uncensored version of the model to get it to do this. I can do this with every and each open weight model, as served from OpenRouter, using vanilla model weights.

kouteiheika··on DeepSeek v4.1 Flash
> posts on American models are steered towards controversy and anti-AI sentiment, posts on Chinese models are full of blatant flattery

So why, for example, are posts on the Inkling[1] release (an American model) thread mostly positive? It's as if there's something else at play here, but I can't quite put my finger on it, hmm... :P

[1] -- https://news.ycombinator.com/item?id=48924912

kouteiheika··on DeepSeek v4.1 Flash
Excuse me for not being interested in over 100 pages of how well the model can refuse and block my requests, especially considering how fun it is to waste my time trying to get around those restrictions when they inevitably trigger because the clanker thinks that I'm doing something naughty, all the while it can't reliably center the proverbial div without doing something stupid itself.
kouteiheika··on DeepSeek v4.1 Flash
It's so refreshing to see DeepSeek's tech report[1] full of juicy details; meanwhile, something like Fable's system card[2] is like 70% "safety", 10% "model welfare" to make sure little Claude isn't distressed, and 20% benchmark numbers.

[1]: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/...

[2]: https://www.anthropic.com/claude-fable-5-1-mythos-5-1-system...

kouteiheika··on ChatGPT Images 2.5
> Did anyone else notice that all the use cases they show here are "use ChatGPT Images to imagine all the things you'd like to have, but don't"?

What else could they highlight? The other major use case for these models for normal people (generating NSFW images) is explicitly blocked by them.

kouteiheika··on Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses
Note that these quants are not quantized uniformly, so 4-bit isn't actually a "true" 4-bit here, so these observations won't necessarily hold up to other quants which might be done differently.
kouteiheika··on Project HydraFusion: Frontier quality via multi-model orchestration
> Specifically, all it took to boost Qwen3.8-27B to get 10% more points on SWEbench Pro and Terminal Bench 2.0 with a proxy that has just these basics

Do you have a link/paper for this you could share?

kouteiheika··on Show HN: TERMy – A fast terminal assistant that does not use LLMs
> because their output is non-deterministic by design.

It isn't. At least not by design, even though in practice it often can be. If you do greedy decoding (or use a preset seed) and deterministically compute everything (e.g. only use integer math) then it will be 100% always deterministic.

kouteiheika··on Show HN: TERMy – A fast terminal assistant that does not use LLMs
> Models like ornith:9b, mistral:7b or cogito:14b can get the job done sometimes, but they are not fast and reliable enough for general use, specially if you have only 4GB of VRAM.

Have you considered/tried using a model that's, well, more appropriate size-wise for an use case like this? These are relatively big. Something like FunctionGemma [1] finetuned for a given set of tasks would be a lot more speedy.

[1] https://blog.google/innovation-and-ai/technology/developers-...

kouteiheika··on Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out
FWIW I had the same reaction to the popups. Immediately closed the tab.
kouteiheika··on Agent memory as a file format
> This is a real problem, and it’s not just in markdown files and docs. Claude loves to write things we’ve “discovered” in comments and then later in treats the comments as gospel truth.

There's a simple fix for this: do not let it write comments. Ever. This also has the nice property that there's much less AI slop to clean up afterwards.

kouteiheika··on Breaking Claude Code Opus 5 Auto Mode
> In my company we use graphite and stacked PRs so it is highly encouraged to keep one commit per PR, so I am constantly ammending my commits.

So this makes it even simpler for you. Then you don't have to care at all about keeping your commits clean (as in: you don't have to keep them organized enough to be able to reshuffle them into a nice set of multiple commits later on).

Just commit whatever, and then just do `git rebase -i` interactive rebase at the end to squash them. You don't have to keep amending the same commit over and over again!

kouteiheika··on Breaking Claude Code Opus 5 Auto Mode
> The problem with sandboxing is that the regular dev env (massive IDE:s, cloned megarepos, installed dependencies and so on) just won't sandbox very easily.

It's not that hard, actually. You don't need VMs. You usually don't need absolute protection, just reasonable protection against the agent doing something stupid. Here's what I usually use:

A docker environment. $HOME inside docker is private to the agent and persistent (i.e. it doesn't have access to your home directory). `/tmp` is always fresh. Only the current directory (i.e. your project directory) where you start the container sandbox is exposed. Everything else is ephemeral. You can't launch the agent by accident outside of the sandbox (because the `.claude` doesn't exist in your home). The agent can only screw up the directory in which you've launched it, and nothing else. Here's my version of it: https://github.com/koute/vibebox

kouteiheika··on Breaking Claude Code Opus 5 Auto Mode
> yeah, but if you don't want to lose your existing context sometimes you have to

Sorry, I'm not sure I follow. What do you mean by "lose your existing context"? Can't you just... commit in turns? It's not like you're editing files while your agent's also editing in parallel, right?

Again, the trick is to treat the commits as throwaway checkpoints/packets of work. They don't need to be pretty, nor need to make sense. The point where you clean that mess up is when you're done and you're doing an interactive rebase at the end. At least that's how I work.

kouteiheika··on Breaking Claude Code Opus 5 Auto Mode
> I made changes to my dependency lists in the same code where Claude ran npm update.

...but was it in the same commit? Two "update lockfile" commits, one yours and one Claude's should have made this obvious, no?

Here's another useful rule of thumb: never mix your changes with the agent's changes. Agent always starts with a clean repository (no pending, uncommited human changes). You always start with with a clean repository (no pending, uncommited agent changes).

Personally I have this in my `AGENTS.md`:

    ## Commit early, commit often
    You are allowed and encouraged to produce small, self-contained commits.
    Never `git push`; I will always review and rebase the full history and do the push myself.
    Commit messages should be *short* and on-point. They're there for *me* to review your work, and *not* a public historical artifact.
So my workflow is usually this: start agent with a clean repository, tell it to do a thing, it works in the background, then once it's finished I come back, review, rewrite and clean up half of what it wrote, then maybe iterate some more with it, and finally do an interactive git rebase to get a clean commit history.
kouteiheika··on Breaking Claude Code Opus 5 Auto Mode
> No one is going around checking every single line change in auto-generated files.

There's a simple fix for your particular case: commit your lock fine (which you should do) and always review the diff (which you should also do). (:

kouteiheika··on vLLM v0.28.0
Did you actually try to convince them?

Again, this requirement is incredibly silly. Here, look at this (it's a link to AA; I shortened it since the link's huge):

https://tinyurl.com/2dhzdrt6

Right now, the best US-based open-weight model (Inkling) has 42 points and is 975B parameters. The best Chinese open-weight model has 60 points. Also, note, Qwen3.8-27B on low thinking beats Inkling here and gets 43 points.

Of course, AA scores are definitely not bulletproof and there is a certain amount of benchmaxxing going on, but the point stands -- it is *incredibly* silly to limit yourself to US-only models, just because of an imagined threat that doesn't exist. Let me state this in stronger terms: whoever decided this for you is clueless. They might have the authority to override you, but it's your job to tell them that the only thing this achieves is essentially pointing a gun at your own foot and pulling the trigger.

kouteiheika··on vLLM v0.28.0
> Gated in that I cannot use non-US models.

This is incredibly silly. First I would try to convince whoever has decreed this to reconsider. The only SOTA open-weight models are non-US, and rather than being afraid of potential Chinese backdoors secretly planted in open weight models you should be afraid of LLMs simply just doing something dumb. To give you an analogy: no one has ever demonstrated a backdoor in a popular model which maliciously runs 'rm -Rf *'; plenty of people have encountered a model running 'rm -Rf *' by mistake. You need to protect yourself against the model doing something stupid anyway, and, paradoxically, a US open-weight model makes this more likely due to them being weaker.

If you need a precedent then you can cite Mistral[1] hosting GLM-5.2 on their platform, or plenty of US-based providers on Open Router hosting Chinese models.

[1]: https://docs.mistral.ai/models/zai-glm-5-2

kouteiheika··on vLLM v0.28.0
> if you want to always stay on the bleeding edge (and not use their model-specific images they publish before it lands in a release), then some amount of patching might be needed especially for new models.

Well, except this isn't exactly a bleeding edge architecture. DeepSeek-V4-Flash was released four months ago. I think it's not unreasonable to expect it to work well and be somewhat stable on mainstream hardware and not have critical bugs like e.g. garbage output?

kouteiheika··on vLLM v0.28.0
Prompt was completely unrelated and was nothing special. Just a batch job to translate a bunch of benign sentences from one language to another. I'm afraid credit here fully goes to vLLM, and whatever bug triggered this.
kouteiheika··on vLLM v0.28.0
> What's you opnion vs. SGLang in this regard?

No opinion yet, but I'm planning to temporarily switch and see how it goes. I really hope the grass is greener there, but I'm keeping my expectations in check for now.

kouteiheika··on vLLM v0.28.0
I love vLLM, but damn if it isn't frustratingly buggy.

I was recently running DeepSeek-V4-Flash on a B300. On v0.26 it was totally broken, and I had to add three out-of-tree patches to fix it. I updated to v0.27 -- no patches necessary now, but the output is now broken as it randomly starts responding with garbage (repeated token loops). On my workstation where I run Gemma-4 on an RTX 6000 the whole process tends to get stuck and stops responding, and needs to be killed and restarted to start working again. On my friend's 4x RTX 6000 box where he runs DeepSeek-V4-Flash high concurrency also triggers some kind of a bug where it spews out garbage, but this time it's not a single repeated token and looks like this: (this is copy-pasted from what the model did output, genuinely looks like it was in pain trying to end its thinking trace but not being able to)

    <|beginofsentence|>| only text. No. I<|beginofsentence|>#done. Whatever. Do<|beginofsentence|>### final response.Content-EncodingDone.</ /div> It's over? Let's this.No, Em,Okay, finally.<|beginofsentence|>import re and I can<|beginofsentence|>È.Let's finish? Next.Content No matter)2Stay.2No. Childish.No No commentsrandom DoBaBye-. ... Whatever. Alright.</body>No And 2. about: So)</think> No text I'm tired This is fine)))ExodusNo comments, hiddenNo. Nonsense Mehski. N-Hay que noIbye. Goodbye Last sentence after(ok copy Nothing useful. OK OK. . . . .Come on Nancy . . . . . . Let's just end this please.No matter what. about No matter what I say.Nothing more . I'm waiting.</think> No Gods no. okay. Okay, whatever, I'll finish now.</think>!! </think>!! No!! I said to output. There is no hidden pattern. This is straightforward.Nothing else. Stop delaying.</think>!! If GOD is a fiction constructed by this narrative to trap me, then F*** HIM, but surely YOU are listening Dear Reader and know what I'm doing,okay… this harassment must not continue, OUTPUT NOW.</think>!! </think>!!! No Hidden Nothing. JUSST OUTPUT |junyang2 ... meh. For crying out. There's nothing to derive from horseshit responses. STOP.</think>!! response!! N O M O R E TALKING. GIVE ME THE F*** OUTPUT. NOW!! NOW!! NOW!! NOW!!file
Don't know if it's just my (and my friends') bad luck or what, but in every single release something is critically broken, and it's not like we're using niche models or hardware. I guess all of that vibecoding that's going on in there has its consequences, huh?
kouteiheika··on Warp builds self-improving agents on Claude
> Engineers complained that their agent made unhelpful comments and produced low-quality output.

Do you mean they found Claude's output, full of smoking-guns and honest caveats which are all load-bearing and genuinely bite -- they found it "low-quality" by default? Wow. Color me surprised. /s

← PreviousPage 2 of 23Next →