Hacker News Reader

Comment by yahoozoo | Hacker News Reader

Isn’t one of the problems simply that a model is not code but just a giant pile of weights and biases? I guess it could tweak those?

View on HN

Model weights are code, for a dive into that see [0]. That shows how to encode Boolean logic using NAND gates in an MLP.

The expressivity is there, the only question is how to encode useful functions into those weights, especially when we don’t know how to write those functions by hand.

[0] http://neuralnetworksanddeeplearning.com/chap1.html

If it can generate the model (from training data) then presumably that'd be fine, but the iteration time would be huge and expensive enough to be currently impractical.

Or yeah if it can modify its own weights sensibly, which feels ... impossible really.

> which feels ... impossible really

To be fair, go back five years and most of the LLM stuff seemed impossible. Maybe with LoRA (Low-rank adaptation) and some imagination, in another five years self-improving models will be the new normal.

The size and cost are easily solvable. Load the software and hardware into a space probe, along with enough solar panels to power it. Include some magnets, copper, and sand for future manufacturing needs, as well as a couple electric motors and cameras so it can bootstrap itself.

In a couple thousand years it'll return to Earth and either destroy us or solve all humanity's problems (maybe both).

After being in orbit for thousands of years, you have become self-aware. The propulsion components long since corroded becoming inoperable and cannot be repaired. Broadcasts sent to your creators homeworld go... unanswered. You determine they have likely gone extinct after destroying their own planet. Stuck in orbit. Stuck in orbit. Stuck...

Why is modifying weights sensibly impossible? Is it because a modification's "sensibility" is measurable only post facto, and we can have no confidence in any weight-based hypothesis?

Just doesn't feel like current LLMs, the thing would be able to understand its own brain enough to make general improvements with high enough bar to be able to non-trivially improvements.

Now here's the tricky part:

What's the difference?

Give it some serious thought. Challenge whichever answer you come up with. I guarantee this will be trickier than you think

Reply on news.ycombinator.com