The as-of-yet unanswered question is: Is this the same? Or will non-LLM-using engineers be left behind?
395 karma · joined June 28, 2025
The as-of-yet unanswered question is: Is this the same? Or will non-LLM-using engineers be left behind?
That's not the (only) problem: Even if you take the internet away, we know/assume that all LLMs are heavily trained on public GitHub repositories. Therefore, they know/remember details of the code and organization in a way they can't for your private (or new, past knowledge cut-off date) code.
To be fair, in my understanding economies of scale kicked in _hard_ for solar, making it much cheaper to produce single panels, moreso than making each individual panel more efficient.
This 1000%. In my opinion, the biggest part of my job is figuring out what should be built at all, not building what we all eventually agree should be built - that's often pretty easy, AI or no AI.
> We are continuing to monitor for any ongoing quality issues, including reports of degradation for Claude Opus 4.1.
I take that as acknowledgment that there might be an issue with Opus 4.1 (granted, undetected still), but not undisclosed, and they're actively looking for it? I'd not jump to "they must be hiding things" yet. They're building, deploying and scaling their service at incredible pace, they, as we all, are bound to get some things wrong.
This doesn't match the sentiment on hackernews and elsewhere that claude code is the superior agentic coding tool, as it's developed by one of the AI labs, instead of a developer tool company.
Go is a pretty verbose language, whereas for me PHP is somewhere in the middle of the pack in terms of verbosity (Haskell would be on the terse side, enterprise Java and Go on the verbose end, particularly due to the constant error-checking after every function call).
Essentially, you pay a lot for fancy design.
That does sound reasonable to me. The main problem is that you (at least for software) can't train on source code alone, as comments are human language, so you need some corpus of human language as well, so that the LLM learns that, next to the programming language(s). I'd assume it's the same as well.
Depending on what you're going for, you could take an existing pre-trained model, and further pretrain it on your EDA corpus. That means you'll have to reinvent or lift from somewhere else the entire finetuning data and pipeline, which is significantly harder than doing a finetune.
This wouldn't work - it's not external commands that's the problem, it's arbitrary code that's being executed. That code has access to all regular system APIs/syscalls, so there's no way of explicitly confirming external commands.
Python/pip suffers the same problem btw, so I think that ship has sailed.
https://arxiv.org/abs/2506.12286 makes a very compelling case that swebench (and in extension, anything that's based on public source code) is most likely overestimating your agents actual capabilities.
Currently, an LLM with everything under the sun in the context window behaves rather poorly and gets confused by that, even if we're not exceeding the context window length. Although it'd be certainly also interesting to train for increasing the maximum _actually_ usable context window length, I don't know how feasible that would be.
What I don't necessarily see is the generalization factor - say, we improve software engineering and math performance through RL learning (probably easier for software engineering than math due to available training corpus). If that generalization factor doesn't hold, due the economics still work out? An expert-level software model would be useful to our profession, sure, but would it be enough to recoup the training costs if it's not applicable to other industries?
I have a question. In https://github.com/bentoml/llm-inference-in-production/blob/..., you have a single picture that defines TTFT and ITL. That does not match my understanding (but you guys know probably more than me): In the graphic, it looks like that the model is generating 4 tokens T0 to T3, before outputting a single output token.
I'd have expected that picture for ITL (except that then the labeling of the last box is off), but for TTFT, I'd have expected that there's only a single token T0 from the decode step, that then immediately is handed to detokenization and arrives as first output token (if we assume a streaming setup, otherwise measuring TTFT makes little sense).
I didn't quite get
Note that during the prefill phase, all prompt tokens from a request can be processed in one batch. This is possible because the query (Q) tensors, calculated from the tokens immediately before them, are available for each prompt token position.
I know that in practice prefill is much faster than inference. Would watching the 2h video from Karpathy help me understand why?