HNHacker News
TopNewBestAskShowJobs

rao-v

1,439 karma · joined October 26, 2024

v@inferencing.net
submissionscomments
rao-v··on LensVLM: Compressing long context as images, expanding only relevant pages
I really like this approach! I sort of think of the vision encoder here as an expensive high fidelity RAG encoder.

The thing I’d love to do with a system like this is train it to be KV cache ordering independent (ie permutation invariant at the page level). Basically each page’s KV cache should be understandable by the model in any ordering - which would allow you to go one step further and treat the KV cache of the vision encoded page as the chunk for the model to reason over.

Then all these zoom in for more detail tricks will extend naturally.

rao-v··on MiMo v2.6
This dashboard is almost certainly built on verl (https://github.com/verl-project/verl), which comes with a bunch of dashboarding capabilities built in (that doesn't look too dissimiliar to these dashboards).
rao-v··on MiMo v2.6
+1 at some point, you need to expect to train a much better base model using everything you've learnt. At the least, you probably want to bring on line the next 10 clever RL environments and ideas your team has been cooking up (which will pipeline into v2.7 etc.)
rao-v··on MiMo v2.6
I might turn this into a blogpost if folks are interested, but my god there is so much clever info in that dashboard.

Here is one really neat bit:

A cutting edge training idea (for agents, it's been used elsewhere for ages) is on-policy RL, basically, it's not enough to say "here is an end to end agentic sequence (including tool calls etc.) that is perfect" you want to say "here is a sequence you might actually have generated that turns out to be correct".

Basically, it's more training efficient to improve models with small tweaks to do more of the right thing they are already doing sometimes than from some perfect oracular "this is the way" answer.

(if you've ever tried to teach humans new skills, you’ve probably noticed this too!)

When you do that, you care about how far the model you are updating (improving) has deviated from the one being used to generate rollouts (agentic rollouts for hard problems can take hours with lots of tool calls, so you can't keep redeploying every slight improvement).

Lo and behold, the dashboard literally has:

partial/avg_staleness (likely the measure of how many micro iterations the "generate answers" model is behind the "improving based on the occasional right answer" model)

train_infer_diff/new_infer/kl (a more direct KL divergence based way of measuring how differently the two models generate tokens)

How cool is that?!

And don't get me started on the clever ideas hiding behind dynsam/avg@n ...

rao-v··on MiMo v2.6
I know we have strong views on what a truly open model is (open weights, open training data, open training code etc.) but I really like how transparent they’ve been about the training of this model.

The realtime dashboard they shared during training (https://mimo.xiaomi.com/rl/) was an incredible learning and teaching tool for me, and they’ve been unusually comprehensive in sharing details about their methodology (check out that tech report - it's got lots of clever behind the scene tricks like Google or Deepseek writeups) and benchmark scores (even the stuff they didn’t do well on).

If you’re releasing an open model going forward, please consider offering the community more of this transparency!

rao-v··on I am often wrong
I find myself slightly more hypothesis-driven, so I tend to be more effective putting much of the focus of 2. Gather missing information after 3. Define the problem ... or at least inside the iteration loop.

As others here have noted, it’s very easy to get lost gathering information that isn’t actually helpful in assessing whether the problem is well formed or whether potential solutions are applicable.

The more you force yourself to specify the problem clearly, but treat your initial problem definition as somewhat suspect - potentially missing key dimensions - the more you will be comfortable finding the information to validate / challenge it (or it's implied solutions) and re-shaping both the problem and it's solutions efficiently.

rao-v··on Hister: A private search engine for the pages you visit and the files you keep
I'd love a extension setting to only send tabs that were visible for ~4+ seconds.

I built myself a little extension last year that tracks what information I was looking at, but focused on generating "new info" recaps for the day / week.

I realized that I open / quick view a lot of pages and close them, which is a strong signal that I don't care about that specific page, and it shouldn't be a source of "new insights" that I learnt that day (since I probably don't care about that topic).

I'd love to re-try a simpler version of that project that builds on Hister as a backend actually.

rao-v··on Bend – a language that blocks AI mistakes via proof and runs on GPUs
Hey Victor! Been following you since HVM/Kind, partly because I'm moderately unhappy with the state of out of the box automatic parallelism in modern languages!

Do you plan to invest in profile guided optimization or autotuning in Bend2 - using runtime profiles / cost models to make decisions around SIMD vs. multicore vs. GPU parallelization?

Bend2's model might give you a really nice view into available parallelization. Heck I can imagine integrating an LLM to profile and optimize in an absurdly expensive `-O7` optimization mode one day!

rao-v··on Xiaomi Mimo 2.6 live post-training dashboard
I absolutely love that someone is doing this! Why isn’t IBM for Granite or Google for Gemini?

If you are going to develop a near frontier model, and you don’t think you have special sauce up your sleeve, why not making training runs and RL environment scores etc. visible to the world?

I’m genuinely learning quite a bit just from the dashboard

rao-v··on Backprop Alternative: Augmented Lagrangian Predictive Coding
Turns out this is more compute intensive than regular backprop. The win (if any) would be in training on future highly distributed architectures where neighbouring parts of the model have accesss to very high bandwidth (to each other) but communicating to further away parts is more expensive
rao-v··on Backprop Alternative: Augmented Lagrangian Predictive Coding
I wonder if you could take a traditional backprop trained LLM and apply this approach to finetuning it (presumably needs less memory and compute?). It could be another entry in the spectrum between LORA and full fine tuning.
rao-v··on Benchmark: CadQuery vs. OpenSCAD for agentic CAD work
I tried modifying the provided CadQuery skill to work with build123d with an agent (surprisingly gemini pretty good at this) and it produced very good results (possibly better than the reported CadQuery/OpenSCAD results after taking a little care to avoid leaking the benchmark pass/fail criteria). Worth a try if you are interested in this space.

I like build123d simply because it can export proper STEP (like CadQuery) but has a nicely python friendly design.

rao-v··on A Design Space Exploration of Async/Await
There is better than even odds I'm older than you, so I'd recommend you rethink using phrases like "possibly longer than you've been alive", it's not ... polite regardless of people's age.

The point (and I'd encourage you to find that thread to not retread ground) is that we absolutely can compile most computation heavy code for these different targets reasonably well - what we cannot garentee is that the resulting code is optimal given context. But gosh we can do so much - I’d encourage you to look into in profile guided, target aware, and autotuning optimization etc. (and then of course, there are LLM guided optimizations, but that's a whole other kettle of fish)

rao-v··on A Design Space Exploration of Async/Await
Threads were historically expensive enough that “just spawn a thread” wasn’t a reasonable thing to do in many situations. Thread pools were sort of a last resort, and we ended up with control flow like objects (futures, await etc.) to multiplex concurrency without parallelism.

Go sort of asks why tho and just standardizes on go routines as a good abstraction over both concurrency and parallelism.

This is sorta true elsewhere too. Go rejects a lot of the machinery that OO languages seem to feel obliged to carry around - inheritance hierarchies, explicit interface implementation etc. For what it's worth, I don't write much go, and I don't think it's magical. I just like how clearly it revisited some basics.

Elsewhere on HA you’ll find my extended rant about how strange it is that we don't have a language that elegantly abstracts computation over threads, SIMD, GPUs etc. Compilers can do this sort of thing now, just not optimally.

rao-v··on A Design Space Exploration of Async/Await
I remember being so mad years ago, coming from a pure CS background, when it dawned on me that async await was “mere” control flow and not actual parallelism.

It’s why I feel go (with go routines being the norm) is one of the few imperative languages that was designed vs. filling out a bunch of historical constraints (apologies this is not meant to trigger a language debate, just an idiosyncratic thought)

rao-v··on I spent $220 on Google app ads and 60% of the installs were robots
Isn't this the sort of thing Google is supposed to be doing for us?
rao-v··on Rune is now open source
I like the idea of making it trivial to work across multiple machines but I’d really prefer not to have to trust your coordination server and encryption approach etc.

Could this (optionally) just run over Tailscale (I suppose ssh is always an option)

https://docs.rune.build/learn/network

rao-v··on DeepSeek v4.1 Flash
umm what are you talking about? Basically this crowd (esp. folks like me who run medium models locally) like open stuff and can be a tiny bit unenthused about opaque mysteries handed down from on high. You'll see people delighted with Gemma releases and heck even IBM's Granite models (boring architecturally though they may be) every time they come out. Heck I was chuffed about gpt-oss-120b for weeks. @sama give us another already!
rao-v··on DeepSeek v4.1 Flash
As I also said on Twitter - it really amazes me how fearless Deepseek are. Every single model release is packed with new and crazy clever ideas and somehow, they always commit to training them at near frontier scale.

I know everybody wants the tell all story of the clever ideas that were developed over the last ~3 years at Anthropic and OpenAI, but what I really want to thumb through is DeepSeek's notebook of "brilliant but didn't quite make the cut" ideas.

They must be trying some truely bonkers stuff to be able to land this much architecture novelty in their full releases.

rao-v··on Training a 3.8B LLM to 0.384 CORE for $998
This is really neat! I'd be tempted to try this again targetting ~1B params and the entire cookbook of "current" small model ideas: gated delta nets (or is ~2K context too short to benefit?), per layer or n-gram embeddings, gated residuals etc.
rao-v··on Factoring RSA 260
Devin … now that’s a name I’ve not hear in a long time
rao-v··on The Navier–Stokes Millennium Prize Problem
It’s reasonable to wonder about what chat usage data gets into models (to be honest probably quite little - carefully curating training data and creating higher quality synth data seems to be the current approach) and the implied risk to privacy and creativity (every new patent filed this year probably touched a model before filing).

What I cannot reconcile is the timeline and the concern in this specific case.

I don’t think training pipelines are anything close to the level of continuous training needed to incorporate Aug 15th ideas into a model that generates a breakthrough early Sept. Either OpenAI nakedly had someone with mathematical understanding dig into a specific user’s chats (a massive red flag) or this really is poor handling of a more classic parallel discovery situation (with one party clearly having worked on it longer)

rao-v··on I tested 10 model/harness combinations on the same Three.js task
For what it’s worth rando redditers warns that cannon.js is old.

I found the remarkable https://babylonjs.com/lite-demos/

Both the regular and lite demos are pretty amazing for browserware!

rao-v··on I tested 10 model/harness combinations on the same Three.js task
Does anyone have an efficient, reasonably designed three.js or other web oriented mini game engine? Low poly but modern rendering effects should be a good niche for hobbyist stuff but hard to find.
rao-v··on I tested 10 model/harness combinations on the same Three.js task
Really appreciate this perspective. It’s tempting to stop at being amazed at what these models can do, but my experience matches yours - they need help with details to do well.
rao-v··on There's a new "Google Jail" for independent wikis
As a bandaid solution we probably just need a good base domain for game wikis attached to a somewhat trustworthy foundation (that won’t sellout to Fandom in a week). Agree that wierdgloop.org is not … great
rao-v··on I tested 10 model/harness combinations on the same Three.js task
It’s really interesting how much little choices make the result better or worse. Astra and one of the GLMs added bright lights, and thus looked so much better to my eye.

Genuinely happy with some of the Qwen 3.8 results (especially since I can run that model at Q8).

Interesting to see how much better (at this task) Pi (OMP) is over Opencode as a harness.

I’d love to see a few more with outcomes that are as easy to judge but less subjective.

I’ve got a toy project going to make a fun to watch battle simulator where an LLM (or two if playing vs) has to write programs that control multiple bots (each with their own line of sight and limited battle context) that have to coordinate and fight alongside each other. Goal is to have the LLM update the code based on current situations maybe 5-10 times in a 5 min simulated battle. Exploring even allow the bots to request new programming and score based on number of reprogram steps.

rao-v··on Formalizing Fermat's Last Theorem
I hear you but point me to one novel formalization right now that is not Lean. It’s really becoming a refacto standard. Which is lovely but terrible for pedegogy
rao-v··on Go grandmaster Shin defeats AI KataGo with a two-stone handicap
Your point is pretty fair.

The reason I think the top 100 pro would win is that when I watch pros review they are very quick to catch when someone is playing a non-current or up to date style. I’ve also seen a few middle game situations where the classic famous go move is now “obviously” not good after we’ve seen how AI handles it.

This all adds up to many ways good sound moves to Lee Changho have now obvious responses. Do I believe he would adapt inhumanly fast? Absolutely! Legendary fighting spirit

rao-v··on Formalizing Fermat's Last Theorem
Right? Might be worth another shot
← PreviousPage 3 of 11Next →