robrenaud

I don't think that's a great analogy.

LoRAs tend to be adapters bolted onto to systems by people other than the system designers, and they are low rank factorizations.

There is nothing low rank or adapter here.

From what I've seen at neurips, in terms of most different but maybe viable, it would be this.

https://sakana.ai/ctm/

In terms of a fresh perspective on designing learning systems, nested learning seems very interesting.

https://abehrouz.github.io/files/NL.pdf

Hearing the clarity, creativity, and force behind his thoughts and speech, I'd give a more than 1/200 chance Ali Behrouz gets himself a Turing award. At the very least, I think he will end making major contributions to AI.

I agree that pass@k feels a bit weird for large k. But for LLMs, it's a decent proxy for "are the knowledge/skills/circuit necessary to solve the problem somewhere in the model". Note that choices for large k is on the order of 256, and the range of valid answers is much larger than that. So your infinite monkeys critique, while true in the limit, wouldn't actually outperform models in the tested regime.

Also, in practice, models don't have that much semantic entropy of a given prompt. With temperature based sampling, models will tend to generate very similar but not identical responses.

RLVR is the more particular term of art in this domain.

VR stands for verified rewards and is the single bit per rollout that is the heart of the post. Maybe we can convince dang to update the title.

He is essentially expanding upon an idea made by Andrej Karpathy on his podcast about a month prior.

Karpathy says that basically "RL sucks" and that it's like "sucking bits of supervision through a straw".

https://x.com/dwarkesh_sp/status/1979259041013731752/mediaVi...

I had some json data that I wanted an annotation interface for. So I asked codex to put it into sqlite and make a little annotation webserver. It worked quickly/easily and without hassle. Sqlite supports queries over json-like objects.

Maybe a very simple document oriented db would have been better?

My biggest gripe is that the sqlite cli is brutally minimal (makes sense given design), but I probably should have been using a nicer cli.

I just wouldn't.

RL is nice in that it is handles messy cases where you don't have per example labels.

How do you build a learned chess playing bot? Essentially the state of the art is to find a clever way of turning the problem of playing chess into a sequence of supervised learning problems.

I worked on a similiar problem about a year ago, on large dense models.

https://www.lesswrong.com/posts/PkeB4TLxgaNnSmddg/scaling-sp...

In both cases, the goal is to actually learn a concrete circuit inside a network that solves specific Python next-token prediction tasks. We each end up with a crisp wiring diagram saying “these are the channels/neurons/heads that implement this particular bit of Python reasoning.”

Both projects cast circuit discovery as a gradient-based selection problem over a fixed base model. We train a mask that picks out a sparse subset of computational nodes as “the circuit,” while the rest are ablated. Their work learns masks over a weight-sparse transformer; ours learns masks over SAE latents and residual channels. But in both cases, the key move is the same: use gradients to optimize which nodes are included, rather than relying purely on heuristic search or attribution patching. Both approaches also use a gradual hardening schedule (continuous masks that are annealed or sharpened over time) so that we can keep gradients useful early on, then spend extra compute to push the mask towards a discrete, minimal circuit that still reproduces the model’s behavior.

The similarities extend to how we validate and stress-test the resulting circuits. In both projects, we drill down enough to notice “bugs” or quirks in the learned mechanism and to deliberately break it: by making simple, semantically small edits to the Python source, we can systematically cause the pruned circuit to fail and those failures generalize to the unpruned network. That gives us some confidence that we’re genuinely capturing the specific mechanism the model is using.

Benchmarks need to change.

There is a 4 choice choice question. Your best guess is the answer is B, at about 35% chance of being right. If you are graded on fraction of questions answered correctedly, the optimization pressure is simply to answer B.

If you could get half credit for answering "I don't know", we'd have a lot more models saying that when they are not confident.

Open source AI is just a lost term. It has been co-opted. If the weights are released, it's open source. Not because that makes sense, not because it's right, but because that's the unfortunate marketting term that has stuck.

The real open source models need a rebrand. Transparent models, perhaps?

LLMs do have some internal representations that predict pretty well when they are making stuff up.

https://arxiv.org/abs/2509.03531v1 - We present a cheap, scalable method for real-time identification of hallucinated tokens in long-form generations, and scale it effectively to 70B parameter models. Our approach targets \emph{entity-level hallucinations} -- e.g., fabricated names, dates, citations -- rather than claim-level, thereby naturally mapping to token-level labels and enabling streaming detection. We develop an annotation methodology that leverages web search to annotate model responses with grounded labels indicating which tokens correspond to fabricated entities. This dataset enables us to train effective hallucination classifiers with simple and efficient methods such as linear probes. Evaluating across four model families, our classifiers consistently outperform baselines on long-form responses, including more expensive methods such as semantic entropy (e.g., AUC 0.90 vs 0.71 for Llama-3.3-70B)

pytorch

The relevance here is pretty weak.

https://sturdystatistics.com/deepdive?fast=0&q=reinforcement...

I think only 1/10 of the articles is really on topic.

LLMs without a search engine attached suck for product reviews.

A lot of employed people like the status quo for the healthcare that they receive.

"In contrast to their largely negative assessments of the quality and coverage of healthcare in the U.S., broad majorities of Americans continue to rate their own healthcare’s quality and coverage positively. Currently, 71% of U.S. adults consider the quality of healthcare they receive to be excellent or good, and 65% say the same of their own coverage. There has been little deviation in these readings since 2001.

Compared with their counterparts, older adults and those with higher incomes register more positive ratings of the quality and coverage of their own healthcare."

https://news.gallup.com/poll/654044/view-healthcare-quality-...

My biggest insight on my (self diagnosed, high functioning) Autism.

For most people, the golden rule works. Treat other people as you want to be treated, and modulo rare interactions with asshole people, you get along well.

For autistic people interacting with most non-autistic people, you've got to use the diamond rule. Treat other people as they want to be treated.

I love when people just straight up tell me directly that I am wrong because of X and Y. Give me Crocker's rules. This works very poorly in most situations, however. My superpower is offending people by telling them my (readily correctible) perception of the truth.

But being wrong can cause downvotes.

The Gemini IMO result used a specifically fine tuned model for math.

Certainly they weren't training on the unreleased problems. Defining out of distribution gets tricky.

The input size to output quality mapping is not linear. This is why we are in the regime of "build nuclear power plants to power datacenters". Fixed size improvements in loss require exponential increases in parameters/compute/data.

I think your core misunderstanding is that you are assuming K calls to generate 1 token is expensive as 1 call to generate K tokens. It is actually much more expensive to generate serially than even in small batches.

I recently got to know someone with a resting heart rate of 45, who will pretty frequently do 8+ mile trail runs and 100 mile bike rides. He is also an amazing cook who makes decadent and delicious foods. He says he consumes 4,000 calories a day. He still has a sizeable belly. I'd say he is at like 10/10 on exercise, but 3/10 on diet. I was surprised at his extreme cardiovascular fitness while still managing to be overfat.

Why are 100s of millions of people using AI if it is providing no value?

It's not even close.

There is no real difference between fine-tuning with and without a lora. If you give me a model with a lora adapter, I can give you an updated model without the extra lora params that is functionally identical.

Fitting a lora changes potentially useful information the same way that fine-tuning the whole model does. It's just the lora restricts the expressiveness of the weight update so that is compactly encoded.

Meta synthetically generated lots of PHP from Python for Llama 3 for training purposes. Meta writes a crazy amount of PHP internally. Translation tends to be way easier than unconstrained generation for LLMs. But if you can translate and filter a large amount of code, you can learn to generate. If you also translate and run the unittests, you get another layer of error checking.

https://arxiv.org/abs/2407.21783

See figure 8.

Human generated tokens contain so much more information per byte than random street view images.

O3 is OpenAI. Street view is Google. I really doubt OpenAI is scraping enormous amounts of random street view images to train their model.

> In this sparsely populated rural area, "I have at least two homes where I have to build a half-mile to get to one house," Mauch said, noting that it will cost "over $30,000 for each of those homes to get served."

Does spending 30k per household connected make any sense?

Find some nerdy social hobby, become part of a community. Board games, killer queen arcade, and indoor rock climbing have all been a bridge to some close friendships for me.

They could provide verbatim snippets surrounded by explanations of relevance.

Instead of the core of the answer coming from the LLM, it could piece together a few relevant contexts and just provide the glue.