Hacker News Reader

Comment by charm137 | Hacker News Reader

I am using it via OpenRouter for going through some Vulkan code. It's a beast - it has no hesitation putting in instrumentation within the code (having a number of render passes) to dump the framebuffer to PNG files and doing pixel-by-pixel comparisons, putting in false colors in shaders to identify bugs etc. Very impressive.

View on HN

That’s something Opus can also do. Perhaps by coincidence I also recently debugged shader bugs where Opus dumped the resulting render to PNGs and then using Python PIL to do comparisons. It purposefully added false colors too.

Suspect there is a training set going around focused on comparing and extracting data in distinct steps from image input. Saw it first on Opus 5 on a backpack construction task to extract and compensate for a reference tape being slightly offset in a LIDAR scan, followed by heat maps for per pixel comparison between the reference and blender render. Opus 5.5 did go a step further in the same, extrapolating offsets to compensate for capture inconsistencies. Some may not even notice this happening as Claude Code steers the model to use the scratchpad hiding much of this work.

Found this especially impressive because backpack material related research has been something I know models can still get caught up on and the models being decent at web research in coding nowadays does absolutely not apply to these task equally Recommending nylon canvas shell suppliers or waxed cotton when PVC coated cotton is explicitly request, that sort of thing. While not perfect in reading coding docs, models are far better there.

Like with front end and code “parsability”, feel like you can very clearly point at which labs use what training set, with many often sharing the same suppliers. Once I noticed that, I became very bullish on specific training sets over emergence as the source for current gains.

Out of curiosity, which harness do you use for this kind of task, and how do you sandbox it? I used Aider before, and consider switching to a more recent tool.

I started out with agy CLI but it's horrible because of its very indeterminate context compaction. But I live with it because it's the only way to use Google's subscription. I was and still use Opencode in parallel (accessing Openrouter ZDR endpoints) when I run out of rate limits / weekly limits.

Over the past two weeks, I have started using a locally modified version of deepseek-harness (modifications include disabling telemetry and querying a provider's model list dynamically through Openrouter, instead of relying on the pre-packaged pi-ai manifest that comes with the base code). So far, that has been the best experience - I really like the caching, tokens/sec stats that it provides. (EDIT: I don't know a lick of TypeScript but AI-driven modification of the harness is good enough for me to make these tweaks).

Sandboxing is a definite issue for me. As you perhaps know already, headless operation is often not an option in graphics/game programming which requires actual render-surface instantiations from windowing systems and then input-context handling (keyboard/mouse/gamepad) as well. So I run most development on a VM. But part of my test-suite and smoke-tests rely on runninng it directly on the host OS (which is not ideal). I'd love to hear suggestions to avoid this.

Can you share your modifications?

Here is the diff patch: https://filebin.net/1wbj38xdt77e048y

You can apply it to the RC2 branch so:

git clone --branch dsh-v0.2.0-rc.2 https://github.com/deepseek-ai/deepseek-harness.git

cd deepseek-harness

git apply /path/to/dsh-v0.2.0-rc.2-to-local.patch

I'd recommend asking the harness to modify itself though (or a cloned repo in another directory) rather than applying the above.

> So I run most development on a VM. But part of my test-suite and smoke-tests rely on runninng it directly on the host OS (which is not ideal). I'd love to hear suggestions to avoid this.

VFIO/GPU passthrough maybe?

Reply on news.ycombinator.com