Hacker News Reader

Comment by GeoAtreides | Hacker News Reader

So you see, if you address this black box in a baby voice, on a Tuesday, during full moon, while standing on one foot, then your chances of a better answer are increased!

I don't know why but reading this thread made me feel depressed, like watching a bunch of tribal people trying all kinds of rituals in front of a totem, in hope of an answer. Say the magic incantation and watch the magic unfurl!

Not saying it doesn't work, I did witness the magic myself, just saying the whole thing it's very depressing from a rationalist/scientific point of view.

View on HN

The use of this sort of anthropomorphic and "incantation" style prompting is a workaround while mechanistic interpretability and monosemanticity work[1] is done to expose the neuron(s) that have larger impacts on model behavior -- cf Golden Gate Claude.

Further, even if end-users only have access to token input to steer model behavior, we likely have the ability to reverse engineer optimal inputs to drive desired behaviors; convergent internal representations[2] means this research might transfer across models as well (particularly, Gemma -> Gemini, as I believe they share the same architecture and training data).

I suspect we'll see understandable super-human prompting (and higher-level control) emerge from GAN and interpretability work within the next few years.

[1]: https://transformer-circuits.pub/2024/scaling-monosemanticit... [2]: https://arxiv.org/abs/2405.07987

I agree. Whatever this is, it's not engineering (not software engineering, anyway), and it does feel like a regression to a more primitive time.

Can ChatGPT Omni read? I can't wait for future people to be illiterate and just ask the robot to read things for them, Ancient Roman slave style.

It reads text from images very well

Isn’t that one of the cornerstones of the Mechwarrior universe, that thousands(?) of years in the future, there is a guild(?) that handles all the higher-level technology, but the actual knowledge has been long forgotten, and so they approach it in a quasi-religious way with chanting over cobbled-together systems or something like that?

(Purely from memory from reading some Mechwarrior books about 30 years ago)

Sounds more like the Adeptus Mechanicus from Warhammer 40K: https://warhammer40k.fandom.com/wiki/Adeptus_Mechanicus

It gets worse if you imagine a future AGI which just tells us new novel implementations of previously unknown physics but it either isn’t willing or can’t explain the rationale.

Reply on news.ycombinator.com