HNHacker News
TopNewBestAskShowJobs

criemen

395 karma · joined June 28, 2025

submissionscomments
criemen··on Getting AI to work in complex codebases
Yet a lot of people never bother to learn vim, and are still outstanding and productive engineers. We're surely not seeing any memos "Reflexive vim usage is now a baseline expectation at [our company]" (context: https://x.com/tobi/status/1909251946235437514)

The as-of-yet unanswered question is: Is this the same? Or will non-LLM-using engineers be left behind?

criemen··on Getting AI to work in complex codebases
What does PRD mean? I never heard that acronym before.
criemen··on CompileBench: Can AI Compile 22-year-old Code?
> because the LLMs are able to look up bug fixes in the project git repository

That's not the (only) problem: Even if you take the internet away, we know/assume that all LLMs are heavily trained on public GitHub repositories. Therefore, they know/remember details of the code and organization in a way they can't for your private (or new, past knowledge cut-off date) code.

criemen··on New thermoelectric cooling breakthrough nearly doubles efficiency
> Solar panels used to be horrible at efficiency. Now they’re pretty amazing and extremely competitive in the power generation market.

To be fair, in my understanding economies of scale kicked in _hard_ for solar, making it much cheaper to produce single panels, moreso than making each individual panel more efficient.

criemen··on The LLM Lobotomy?
Batch size is dynamic, in MoE apparently the experts chosen depend on the batch (not only your single inference request, which sounds weird to me, but I'm just an end user), no one audited the inference pipeline for floating point nondeterminisms, and I'm not even sure that temperature 0 implies deterministic sampling (the quick math formula I found has e^(1/temp) which means that 0 is not a valid value anyways and would need some dealing with).
criemen··on The LLM Lobotomy?
Even with temperature 0, the LLM output will not be deterministic. It will just have less randomness (not defined precisely) than with temperature 1. There was a recent post on the frontpage about fully deterministic sampling, but it turns out to be quite difficult.
criemen··on An open-source maintainer's guide to saying “no”
> Even before AI, i'd argue the vast majority of code is cheap and simple. But that is what makes it more important than ever to decide what code should exist before someone wastes a day or week writing it.

This 1000%. In my opinion, the biggest part of my job is figuring out what should be built at all, not building what we all eventually agree should be built - that's often pretty easy, AI or no AI.

criemen··on Claude now has access to a server-side container environment
They posted

> We are continuing to monitor for any ongoing quality issues, including reports of degradation for Claude Opus 4.1.

I take that as acknowledgment that there might be an issue with Opus 4.1 (granted, undetected still), but not undisclosed, and they're actively looking for it? I'd not jump to "they must be hiding things" yet. They're building, deploying and scaling their service at incredible pace, they, as we all, are bound to get some things wrong.

criemen··on A staff engineer's journey with Claude Code
> The people who build the models don't understand how to use the models. It's like asking people who design CPUs to build data-centers.

This doesn't match the sentiment on hackernews and elsewhere that claude code is the superior agentic coding tool, as it's developed by one of the AI labs, instead of a developer tool company.

criemen··on Anthropic raises $13B Series F
I'm not so confident in that yet. If you look at the inference prices Anthropic charges (on the API) it's not a race to the bottom - they are asking for what I feel is a lot of money - yet people keep paying that.
criemen··on The Rise of Hybrid PHP: Blending PHP with Go and Rust
I'm not so sure: If you think about hiring talent, there's a perception in the wider industry about PHP (rightly or wrongly, let's not debate that here) that'll prevent access to a lot of great developers. So even if not warranted anymore on technical merits anymore, in terms of employer branding, I'd say migrating away from PHP is helping.
criemen··on The Rise of Hybrid PHP: Blending PHP with Go and Rust
How did you reduce LoC with a move from PHP to go?

Go is a pretty verbose language, whereas for me PHP is somewhere in the middle of the pack in terms of verbosity (Haskell would be on the terse side, enterprise Java and Go on the verbose end, particularly due to the constant error-checking after every function call).

criemen··on It is worth it to buy the fast CPU
This desk (used to?) fit the budget, for example: https://www.architonic.com/en/p/holmris-b8-milk-classic-1070...

Essentially, you pay a lot for fancy design.

criemen··on Best Options for Using AI in Chip Design
> Indeed perhaps it's important to include a high quality corpus in pre training? I doubt anyone wants to train an LLM from scratch for EDA.

That does sound reasonable to me. The main problem is that you (at least for software) can't train on source code alone, as comments are human language, so you need some corpus of human language as well, so that the LLM learns that, next to the programming language(s). I'd assume it's the same as well.

Depending on what you're going for, you could take an existing pre-trained model, and further pretrain it on your EDA corpus. That means you'll have to reinvent or lift from somewhere else the entire finetuning data and pipeline, which is significantly harder than doing a finetune.

criemen··on How we exploited CodeRabbit: From simple PR to RCE and write access on 1M repos
> Maybe those tools should explicitly confirm executing every external command

This wouldn't work - it's not external commands that's the problem, it's arbitrary code that's being executed. That code has access to all regular system APIs/syscalls, so there's no way of explicitly confirming external commands.

Python/pip suffers the same problem btw, so I think that ship has sailed.

criemen··on Does the Bitter Lesson Have Limits?
All links render as blue strike-through line in Firefox (underline in Chrome), hurting legibility :(
criemen··on Atlassian terminates 150 staff
Presumably because the layoff is targeted at a specific (set of) teams in customer service, rather than a 1% haircut across the company?
criemen··on Gemini 2.5 Deep Think
How much would it cost at list-price API pricing?
criemen··on Principles for production AI agents
Running evals aren't the problem, the problem is acquiring or building a high-quality, non-contaminated dataset.

https://arxiv.org/abs/2506.12286 makes a very compelling case that swebench (and in extension, anything that's based on public source code) is most likely overestimating your agents actual capabilities.

criemen··on Visa and Mastercard: The global payment duopoly (2024)
Visa and Mastercard have a net profit margin in the order of 50%. Do you think that's justified for the service they're providing, and is indicative of a healthy market/competition?
criemen··on Amazon and the “Profitless Business Model” Fallacy
Thanks for the reminder! https://www.youtube.com/watch?v=TjDEsGZLbio
criemen··on Mistral Releases Deep Research, Voice, Projects in Le Chat
Perplexities isn't bad? Although I lack the OpenAI subscription to compare.
criemen··on Mistral Releases Deep Research, Voice, Projects in Le Chat
I totally get it. Due to my work, I mostly keep up with new model releases, but the pace is not sustainable for individuals, or the industry. I'm hoping that model releases (and the entire development speed of the field) will slow down over time, as LLMs mature and most low-hanging fruits in model training have been picked. Are we there yet? Surely not.
criemen··on The upcoming GPT-3 moment for RL
I'd expect that's manageable by some sort of agent-of-agent pattern. You have a high-level planning instance that calls upon fresh LLM instances (new context window!) for executing more targeted tasks or bug-fixes.

Currently, an LLM with everything under the sun in the context window behaves rather poorly and gets confused by that, even if we're not exceeding the context window length. Although it'd be certainly also interesting to train for increasing the maximum _actually_ usable context window length, I don't know how feasible that would be.

criemen··on The upcoming GPT-3 moment for RL
My understanding is that this is essentially how RLHF works, and it doesn't scale. As you run RL for longer, the model will learn how to cheat the imperfections of the grader, instead of getting better at the task at hand. Therefore, to scale RL you really need good graders, and determinism is king.
criemen··on The upcoming GPT-3 moment for RL
They have a point about RLs increasing importance. From my outsider perspective, all major advances in model capabilities in the last period of time come from RL, so it's natural to expect that we can "milk" RL more for performance gains. Scaling RL is a natural way to attempt that.

What I don't necessarily see is the generalization factor - say, we improve software engineering and math performance through RL learning (probably easier for software engineering than math due to available training corpus). If that generalization factor doesn't hold, due the economics still work out? An expert-level software model would be useful to our profession, sure, but would it be enough to recoup the training costs if it's not applicable to other industries?

criemen··on LLM Inference Handbook
Thanks a lot for putting this together!

I have a question. In https://github.com/bentoml/llm-inference-in-production/blob/..., you have a single picture that defines TTFT and ITL. That does not match my understanding (but you guys know probably more than me): In the graphic, it looks like that the model is generating 4 tokens T0 to T3, before outputting a single output token.

I'd have expected that picture for ITL (except that then the labeling of the last box is off), but for TTFT, I'd have expected that there's only a single token T0 from the decode step, that then immediately is handed to detokenization and arrives as first output token (if we assume a streaming setup, otherwise measuring TTFT makes little sense).

criemen··on Life of an inference request (vLLM V1): How LLMs are served efficiently at scale
And on the topic of prefill: Do you know what the role of GPUs is vs. in inference?
criemen··on Life of an inference request (vLLM V1): How LLMs are served efficiently at scale
Thanks for writing the article!

I didn't quite get

Note that during the prefill phase, all prompt tokens from a request can be processed in one batch. This is possible because the query (Q) tensors, calculated from the tokens immediately before them, are available for each prompt token position.

I know that in practice prefill is much faster than inference. Would watching the 2h video from Karpathy help me understand why?

← PreviousPage 5 of 5