BenoitP

For context: Goodfire has expertise in interpretability through a Sparse Auto Encoder technique of their own

https://www.goodfire.com/blog/a-practical-guide-to-saes

To me their purpose only came clearly recently. They're just a way to rotate smoothly one dimension into another. And control the speed of the rotation.

x into y. Potential energy to kinetic energy. Charge into current.

Keeping this definition but going higher in the dimensions: x into y, y into z and z into x. Going meta, the x into y rotation defines a plane; And a plane going into another direction is a volume. (x into y) into z.

What made it click, what made it beautiful is Geometric Algebra:

https://youtu.be/60z_hpEAtD8?is=e3GxYU4J1SdpsumW

Always have been, for higher-level languages.

James Gosling, 1997:

> The feel of Java

> Java is a blue collar language. It’s not PhD thesis material but a language for a job. Java feels very familiar to many different programmers because we preferred tried-and-tested things.

https://horstmann.com/corejava/feel-of-java.pdf

And now we're all witnessing a major caveat of LLMs: the burden of verification is pushed to the reviewers, while the proposer will get all credit.

Average user input quality is not optimal by definition. SO's upvote system and reputation system is what really refined quality. Much like Reddit's system is helping a lot for conversation. Much like academia's publish and citation system is an even better system.

I'm not saying LLM won't be able to train further, but tough and sound reviews are paramount to improving the corpus of quality content. And with the replacement of SO by LLMs, one can wonder if one quality system went down the drain. Anyway there's still code reviews on GitHub and in private repos, but we're still far from getting the human judgement out of the loop.

Oil is sky high, a recession might be near. Also, optimizations (like the current KV one) can yield a drastic reduction in RAM needs. It's a risk.

If I was trying so stabilize (just stabilize) funding for humongously expensive fabs: I'd be giving markets reassurances that my business will continue to be high margins for the mid-future.

What an eerie feeling reading this, or was it prescient?:

"""

    def fizz_buzz_encode(i):
        if   i % 15 == 0: return np.array([0, 0, 0, 1])
        elif i % 5  == 0: return np.array([0, 0, 1, 0])
        elif i % 3  == 0: return np.array([0, 1, 0, 0])
        else:             return np.array([1, 0, 0, 0])

interviewer: OK, that's probably enough.

me: That's enough setup, you're exactly right. [<--- !!] [...]

"""

Damn, Claude was there all along

Since this is about Geometric Algebra, here are two other (complementary) great intros:

* Why can't you multiply vectors? - Freya Holmer - https://www.youtube.com/watch?v=htYh-Tq7ZBI&t

* A Swift Introduction to Geometric Algebra - sudgylacmoe - https://www.youtube.com/watch?v=60z_hpEAtD8

Compare-and-swap is a hardware instruction, and beyond that a desirable pattern for dealing with concurrent writes.

CMPXCHG

https://www.felixcloutier.com/x86/cmpxchg

https://lwn.net/Articles/847973/

There's even a load-bearing in there as well

And GLM local models can eat Anthropic's lunch just like both eat OpenAI's.

Mistral's revenue is mostly B2B, and that's much more difficult to move.

Haha :)

Well, that was the joke. If you're reading this, hi Ron!

It helps you spend more token, is more expensive, and thus is obviously more AI. Also novelty and more complexity means less scrutiny of the approach.

These are necessary and perfectly sufficient for an investment firm thesis I believe.

> That has not been the case, and I no longer think that it will be. People get by with the existing programming languages and models, that seems to be fine.

.. as long as the processing they do fits in a map().

I still deeply believe that a new class of programs can be unlocked by using reduce() or all_gather() in a massive concurrent stream.

In a stadium, why can't a uuid be embedded as an acoustic chirp emitted by all the smartphones, then triangulation-reduced in a 3D map of all devices, enabling a stadium-wide sound system?

Why don't we have MMORPGs that can deal with 100k users in a single game instance?

> Is my friend a virus for recommending me their favorite book?

In that case the book would be a virus, and your friend a host.

But it generalizes way wider to ideas. Religions are viruses too for example. With very strong viral mechanisms sometimes. Some religions are so much entrenched so that they have become part of the fabric for entire societies or countries, morphing their culture; the same way that sometimes viral RNA can become part of the DNA of the host.

Another project worth checking would be Quasar (and had virtual threads in 2013):

https://docs.paralleluniverse.co/quasar/

But I believe the author moved to a Big Corp which sells databases, working on some big project which took all his time...

France exports about 60TWh of electricity yearly for about 4B€.

Flamanville running at 100% generates 32TWh. It is amortized in 9 years. I'd say it's doing pretty okay even with its ballooning cost.

Took your link, changed de to en, reduced granularity weekly to daily. It looks way more spiky:

https://www.energy-charts.info/charts/energy/chart.htm?l=en&...

Also, both wind and solar (especially solar) change vastly under a sub-daily aggregation.

It'd be really great if wind turbines across Germany were producing at uncorrelated times.

Let's look at the wind right now: uniform wind across Germany:

https://earth.nullschool.net/#2026/09/04/1200Z/wind/surface/...

Let's look at the wind earlier: uniformly no wind across Germany:

https://earth.nullschool.net/#2026/09/02/0000Z/wind/surface/...

Hmmmm.

Diesel, or rather energy, is also a way bigger part of every single aspect of our modern world. Let's take drinking a can of coke as an example:

First degree: It came into your home from the factory through the shop. 80% product + 20% diesel.

Second degree: Product is aluminum can + water + ingredients. Aluminium came to be a can through 70% refined aluminum + 15% diesel (transport) + 15% electrical energy (pressing the can). Water was pumped locally: free from the ground + 100% electrical energy. Ingredients came from a farm: 80% local product + 20% diesel.

Third degree: Electrical energy came from 40% gas turbines, 30% hydro, 30% wind farm. Ingredients from farms came from soil 40%, 35% nitrogen fertilizer (== gas), 25% diesel (machines)

... and so on

If you decompose everything all the way, the global tally is either matter, which is given to us by nature for free. Or energy; The sources of which being vastly composed of fossil fuels. Fossil fuel prices affect everything.

These fossil resources being finite we can also conclude we're experiencing a centuries-long hay-fire. But hey, we're merely plateauing right now; and most of us won't be here for the harder part of the crash.

And I didn't even mention climate change.

What's wrong with https://ann-benchmarks.com/index.html ?

They had vertical integration and made bank. Then each layer got more complex as computing became mainstream, and IBM got out-competed on everyone of them by specialized companies. And then they sold them off. The only layer that remains is the most strategic: the customer relationship.

IBM is now just a consultancy.

To the more exercise-inclined, there's this introductory game:

https://incredible.pm/

Same here

A potential solution would be to have a reputation system, with some light pageranking in it. People who care about their reputation are bound to rate other likewise people higher. I don't think such loyalty exists between vibetalkers.

It's obvious we're not alone, and 2 distinct clusters would form (with people on a spectrum in between)

Not doing retrieval, but one thing I'd love to try would be active learning passes of querying. Here is a paper for large scale active learning

https://cdn.aaai.org/ojs/20591/20591-13-24604-1-2-20220628.p...

Steps would be:

- A first hybrid query with bm25+query vector

- rank some first results locally (ask the LLM for relevance, usefulness? ask the human?)

- query in active learning manner; providing positive and negative examples to orient the direction of better results. It'd silence the polluting components from the original query

- iterate

If anyone's hiring for remote roles, I'd love to give that technique a try

Yet another manycore proposal, but I feel each time we're getting closer. Bandwidth is one of the few dimensions still growing, and there's something to unlock by shaping computation by this firehose of bits.

However it seems the software part is always a blocker, and these architecture only address well a handful of program types.

Is this time different? I think it is. The paradigm here is about small threads that yield very often, and we have that way of programming: Erlang, Go, Java's virtual threads.

LCOE is good for marginal cost (eg: one more solar panel), but fails dramatically at evaluating systemic costs.

A nuclear reactor moves the entire market down, including the costs to the consumer when he buys solar energy.

Here is a UN document explaining it: https://unece.org/sites/default/files/2025-09/GECES-21_2025_...

JAX is designed from the start to fit well with systolic arrays (TPUs, Nvidia's tensor cores, etc), which are extremely energy-efficient. WebGL won't be the tool that connects it on the web, but the generation after WebGPU will.

I believe we could get there eventually. For example for collision there is work to make it differentiable (or use a local surrogate at the collision point): https://arxiv.org/abs/2207.00669

The robotics will need to connect vision with motors with haptics with 3D modelling. And to propagate gradient seamlessly. For calibrating torque with the the elastic deformation of the material for example. After all matter is not discreet at small scales (staying above the atomic scale)

All this will require all modules to be compatible with differentiability. It'll be expensive at first, but I'm sure some optimizations can get us close to the discreet case.

Also even for meshes there is a lot to gain with trying to go the continuous way:

https://www.cs.cmu.edu/~kmcrane/Projects/DDG/

Yeah :)

I had a lot of fun writing the article! And it is only half a joke

My intuition for so-called world models is that we'll have to plug modules, each responsible for a domain (text, video, sound, robot-haptics, physical modelling) It'll require to plug modules in a way that will allow the gradient to propagate. A differentiable architecture. And JAX seems well placed for this by making function manipulation a first citizen. Looking at your testimony comforts me in this view