Someone else asked something similiar but basically, how does / should this work when candidate drop targets are many e.g. when dragging a first or second rectangle onto a page in Figma when no relationships have obviously been set.
1,358 karma · joined October 26, 2024
They should be doing what this test did, and generalizing and significantly varying the training environments (and ensuring some of them are unexpectedly complex and unexpectedly simple).
Further I suspect the smaller models are especially overfitting to these sorts of simpler RL envs (benchmarks tend to be much simpler than the hardest RL env used today) because they are more likely to be scoring at ~20-40% success rate (which is sort of what you want to get good training signal when doing RL runs).
My sense so far is it’s very much about problem domain “taste” and there is no one correct answer (that is token cost efficient).
Does the smartest agent need the full flavor of the context of the problem to make real progress (i.e what’s been tried, how you’ve influenced the approach) or does the smartest agent really need a crisp problem definition and context room to really work.
Is it just me, or is this late stage token-utopia derangement? It took me quite a few tries before I could put Blender to any actual use because of how ... unusual ... the UI was.
Something like an extra string running along the edge of a wall that could “gather up” the diagonal strings and tuck them away on an edge or corner would make this a much easier impulse purchase.
Its not natively reading the weights in the Matrix “symbols scrolling on a screen” sense. Still bloody impressive and not an outcome I would have bet on!
Looks like this might actually be model weights of some kind, hope someone messes with it (safely) and reports back.
The models are so impressive that getting 70-80% of their capability can be helpful, but beware, you are using a significantly dumber model.
I'll put your knowledge graph nn idea into the mix of things to try.
The model already ingests text, so you just to attach a tiny VLM head that reviews the image and read / writes scene continuity text tokens.
Once you have that you can probably just RL with scoring on continuity issues and eventually the VLM head will learn to update dense continuity tokens?
So many fun wierd experiments to run if you have some compute to mess with. There are competent open video diffusion models that you could test this concept easily enough.
Given that models learn to use residual connections optimally, it can’t be hard for them to steer around the fixed computation layer when it’s not needed.
I’m sure giant models invest a few layers in reproducing these functions, but surely smaller models can benefit?
FWIW:
It looks like the Neanderthals had javelins as ranged weapons: https://www.mdpi.com/2571-550x/6/1/17
The paper I linked shows that we were in and out of Neanderthal territories (or vice versa) a bunch.
Punctuated equilibrium (is that still in fashion?) suggests that the inbreeding / limited population eras of humanity are when we got a lot of our special capabilities!
I know Anthropic are brilliant at knowing when to focus where, but I genuinely expected them to be talking about 3-5x more serious scaled up drug discovery plays at this stage.
Is this not a focus, or did the big plays not want to be part of a roundup post?
You are also rolling the training of your models by size and the lat est small model can sometimes beat an older model.
For a while we thought you just train the big model first then distill smaller models from it but since we've learnt the importance of on policy RL (basically the way a small model figures out a problem is very diff from a big model so it's best to RL optimize each model distinctly)
Reactive optimizers could actually be helpful if you used them to dynamically tweak training data mix and use them to make choices about RL environments (if you recall from the Mimo training dashboards, it’s quite tricky to figure out how to value and chose RL traces during post training)