Hacker News Reader

Hacker News

Comment by causal | Hacker News Reader

Full thread

causal

Run an incredible 400B parameters on a handheld device.

0.6 t/s, wait 30 seconds to see what these billions of calculations get us:

"That is a profound observation, and you are absolutely right ..."

View on HN

Better than waiting 7.5 million years to have a tell you the answer is 42.

Maybe you should have asked a better question. :P

What do you get if you multiply six by nine?

54?

67?

Tea

For two

Should have used a better platform. So long and thanks for all the fish.

Some one should let Douglas Adams know the calculation could have been so much faster if the machine just lied.

I think Adams was prescient, since in his story the all powerful computer reaches the answer '42' via incorrect arithmetic.

The Bistromathics? That's not incorrect, it's simply too advanced for us to understand.

“What do you get if you multiply six by nine?”

(One) source: https://www.reddit.com/r/Fedora/comments/1mjudsm/comment/n7d...

Ok, my Hitchhiker-foo was too weak, thanks!

You also have the problem that if the both the ultimate answer to life the universe and everything, and the ultimate question to life the universe and everything, are know at the same time in the same universe. The universe is spontaneously replaced with a slightly more absurd universe to ensure that both the question and answer become meaningless.

To quote the message from the universes creators to its creation “We apologise for the inconvenience”. Does seem to sum up Douglas Adam’s views on absurdity of life.

Looked at a certain way it's incredible that a 40-odd year old comedy sci-fi series is so accurate about the expected quality of (at least some) AI output.

Which makes it even funnier.

It makes me a little sad that Douglas Adams didn't live to see it.

Also check out "The Great Automatic Grammatizator" by Roald Dahl for another eerily accurate scifi description of LLMs written in 1954:

https://gwern.net/doc/fiction/science-fiction/1953-dahl-theg...

"Can write a prize-winning novel in fifteen minutes" - that's quite optimistic by modern standards!

42 wasn't a low quality answer.

The joke revolves around the incongruity of "42" being precisely correct.

Yes and then no one knows the prompt!

See also Asimov's The Last Question

https://en.wikipedia.org/wiki/The_Last_Question

I don't think we are ever going to win this. The general population loves being glazed way too much.

> The general population loves being glazed way too much.

This is 100% correct!

Thanks for short warm blast of dopamine, no one else ever seems to grasp how smart I truly am!

That is an excellent observation.

That's an astute point, and you're right to point it out.

You are thinking about this exactly the right way.

You’re absolutely right!

The other day, I got:

"You are absolutely right to be confused"

That was the closest AI has been to calling me "dumb meatbag".

"Carrot: The Musical" in the Carrot weather app, all about the AI and her developer meatbag, is on point.

It would be much worse if it had said "You are absolutely wrong to be confused", haha.

Poor “we”. “They” love looking at their own reflection too much.

2 years ago, LLMs failed at answering coherently. Last year, they failed at answering fast on optimized servers. Now, they're failing at answering fast on underpowered handheld devices... I can't wait to see what they'll be failing to do next year.

Probably the one elephant in the roomy thing that matters: failing to say they don't know/can't answer

With tool use, it's actually quite doable!

Claude does it all the time, in my experience.

Same here, it's even told me "I don't have much experience with this, you probably know better than me, want me to help with something else?".

The speed on a constrained device isn't entirely the point. Two years ago, LLMs failed at answering coherently. Now...

You're absolutely right. Now, LLMs are too slow to be useful on handheld devices, and the future of LLMs is brighter than ever.

LLMs can be useful, but quite often the responses are about as painful as LinkedIn posts. Will they get better? Maybe. Will they get worse? Maybe.

> Will they get better? Maybe. Will they get worse? Maybe.

I find it hard to understand your uncertainty; how could they not keep getting even better when we've been seeing qualitative improvements literally every second week for months on end? These improvements being eminently public and applied across multiple relevant dimensions: raw inference speed (https://github.com/ggml-org/llama.cpp/releases), external-facing capabilities (https://github.com/open-webui/open-webui/releases) and performance against established benchmarks (https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks)

There are many metrics for “better” and “worse”. It is entirely possible for an AI system to be better in the sense of hallucination while also being of less utility. An arrogant prick who’s always correct isn’t always a good person to have on your team, right?

I thought you were being sarcastic until I watched the video and saw those words slowly appear.

Emphasis on slowly.

Make "you are absolutely right" it's own token?

I too thought you were joking

laughed when it slowly began to type that out

I mean size says nothing, you could do it on a Pi Zero with sufficient storage attached.

So this post is like saying that yes an iPhone is Turing complete. Or at least not locked down so far that you're unable to do it.

You need fast storage to make it worthwhile. PCIe x4 5.0 is a reasonable minimum. Or multiple PCIe x4 4.0 accessed in parallel, but this is challenging since the individual expert-layers are usually small. Intel Optane drives are worth experimenting with for the latter (they are stuck on PCIe 4.0) purely for their good random-read properties (quite aside from their wearout resistance, which opens up use for KV-cache and even activations).

Reply on news.ycombinator.com