Ironically, the same could be said about Attention Is All You Need in 2017. It didn’t drive any improvements immediately- actual decent Transformer models took a few years to arrive after that.
3,007 karma · joined October 23, 2012
Ironically, the same could be said about Attention Is All You Need in 2017. It didn’t drive any improvements immediately- actual decent Transformer models took a few years to arrive after that.
Again, nobody thinks trees in computer science contains squirrels, nobody thinks airplanes are birds, and nobody thinks a neuron in a ML model contains axons and dendrites. This is a weird hill to die on.
Are you gonna complain that the word “photograph” is “light writing” but in reality nobody is writing anything so therefore the word is wrong?
2. You are about 5 years behind in terms of the research. Look into hierarchical feature representation and how MLP neurons work. (Or even in older CNNs and RNNs etc). And I'm willingly using the word "neuron" instead of "feature" here because while I know "feature" is more correct in general, there are definitely small toy models where you can pinpoint an individual neuron to represent a feature such as a face.
We can trace which neurons activate for a face recognition model and see that a certain neuron does light up when it sees a face. The correct features are active for the sentence “the word parrots is plural”.
If you stop assuming the LLMs have no internal representations of the data, then everything makes a lot more sense! The LLM is FORCED to answer questions… just like a high school student filling out the SAT is forced to answer questions.
If a high school student fills out the wrong answer on the SAT, is that a hallucination?
Hallucinations are expected behaviors if you RLHF a high schooler to always guess an answer on the SAT because that’ll get them the highest score. This applies to ML model reward functions as well.
Come on guys, Debian 13 has been in testing for months, and you can't be arsed to update your apt repos from bookworm to trixie by release, or even weeks after release? That's embarrassing.
~ sudo apt update --audit
[...]
Hit:8 https://packages.cloud.google.com/apt google-compute-engine-bookworm-stable InRelease
Hit:10 https://packages.cloud.google.com/apt cloud-sdk-bookworm InRelease
Hit:11 https://pkg.cloudflareclient.com bookworm InRelease
Hit:12 https://pkg.cloudflare.com/cloudflared bookworm InRelease
[...]
Fetched 407 kB in 2s (222 kB/s)
2 packages can be upgraded. Run 'apt list --upgradable' to see them.
Warning: https://pkg.cloudflare.com/cloudflared/dists/any/InRelease: Policy will reject signature within a year, see --audit for details
Audit: https://pkg.cloudflare.com/cloudflared/dists/any/InRelease: Sub-process /usr/bin/sqv returned an error code (1), error message is:
Signing key on FBA8C0EE63617C5EED695C43254B391D8CACCBF8 is not bound:
No binding signature at time 2025-08-21T15:58:52Z
because: Policy rejected non-revocation signature (PositiveCertification) requiring second pre-image resistance
because: SHA1 is not considered secure since 2026-02-01T00:00:00Z
These apt repos are still bookworm-only after the trixie release, and it's been weeks. And Cloudflare is still stuck on SHA1.Well, it's a shit edited article, but at least it's not AI shit
If someone smokes a lot of cigarettes and gets lung cancer, we don’t blame the doctor for the lung cancer. This shit is the patient’s responsibility.
It'll compile fine against latest CUDA 12.8 or 12.9 etc, but there's zero need to pack whatever the latest CUDA version is.
True
> then you add just a single neuron that recognizes them perfectly
Not true.
Don't think in terms of neurons, think in terms of features. A feature can be spread out over multiple neurons (polysemanticity), I just use a single neuron as a simplified example. But if those multiple neurons perfectly describe the feature, then all of them are important to describe the feature.
The Universal Approximation Theorem implies that a large enough network to perfectly achieve that goal would exist (let's call it size n or larger), so eventually you'd get what you want between 0 and n neurons.
https://www.youtube.com/watch?v=AgkfIQ4IGaM
That's not a mirage, it's clearly capability that a smaller model cannot demonstrate. A model with less parameters and less hidden layers cannot have a neuron that lights up when it detects a face.
llamacpp> ls -l \*llama\*
-rwxr-xr-x 1 root root 2505480 Aug 7 05:06 libllama.so
-rwxr-xr-x 1 root root 5092024 Aug 7 05:23 llama-server
That's a terrible excuse, Llama.cpp is just 7.5 megabytes. You can easily ship a couple copies of that. The current ollama for windows download is 700MB.I don't buy it. They're not willing to make an 700MB download a few megabytes bigger to ~730MB, but they are willing to support a fork/rewrite indefinitely (and the fork is outside of their core competency, as seen by the current issues)? What kind of decisionmaking is that?
What's wrong with using an older well-tested build of llama.cpp, instead of reinventing the wheel? Like every linux distro ever who's ever ran into this issue?
Red Hat doesn't ship the latest build of the linux kernel to production. And Red Hat didn't reinvent the linux kernel for shits and giggles.
There's a post on github of a madman who got llama.cpp generating tokens for an AI model that's running on an Intel Arc, Nvidia 3090, and AMD gpu at the same time. https://github.com/ggml-org/llama.cpp/pull/5321
That's pretty small.
Even Deepseek R1 0528 685b only has like ~16GB of attention weights. Kimi K2 with 1T parameters has 6168951472 attention params, which means ~12GB.
It's pretty easy to do prompt processing for massive models like Deepseek R1, Kimi K2, or Qwen 3 235b with only a single Nvidia 3090 gpu. Just do --n-cpu-moe 99 in llama.cpp or something similar.
The AMD AI MAX 395+ gives you 256GB/sec. The M4 gives you 120GB/s, and the M4 Pro gives you 273GB/s. The M4 Max: 410GB/s (14‑core CPU/32‑core GPU) or 546GB/s (16‑core CPU/40‑core GPU).
So the CPU and RAM are soldered on the motherboard.
You can buy just the motherboard from Framework: https://frame.work/products/framework-desktop-mainboard-amd-...
That's the exact perfect use case for Docker, versus something heavier weight like a VM. What, you expect generic enterprise code to somehow be too dangerous for Docker but acceptable in a VM?
I'd still take a Docker container over an Apple container, because even though docker is not VM-level-secure, it's good enough for running local AI generated code. You don't need DEFCON Las Vegas levels of security for that.
And also because Docker runs on my windows gaming machine with a fast GPU with WSL ubuntu, and my linux VPS in the cloud running my website, etc etc. And most people have already memorized all the basic Docker commands.
This would be a LOT better if it was just a single docker command we can copy paste, run it a few times, and then delete if necessary.
Similar to a `docker compose up -d` that a lot of projects offer. Just download the docker-compose.yml file into a folder, run the command, and you're running. If you want to delete everything, just `docker compose down` and delete the folder, and the container and everything is gone.
Anything similar to that? I don't want to run a random install.sh on my machine that does god knows what.
From there, statistics comes into play. There's just a bigger pool of Indians to select CEOs from.
Am I supposed to spend money on Amazon.com when I visit the website just because Amazon wants me to?
Prejudice is a bad thing- for things that people can't change, like their race or age. Prejudice against people making bad or wasteful decisions is a good thing.
That port right next to the water has probably disappeared.