HNHacker News
TopNewBestAskShowJobs

jychang

3,007 karma · joined October 23, 2012

submissionscomments
jychang··on From multi-head to latent attention: The evolution of attention mechanisms
> What's the value of "pointing out limitations" if this completely fails to drive any improvements?

Ironically, the same could be said about Attention Is All You Need in 2017. It didn’t drive any improvements immediately- actual decent Transformer models took a few years to arrive after that.

jychang··on Some thoughts on LLMs and software development
https://en.m.wikipedia.org/wiki/Artificial_neuron
jychang··on Some thoughts on LLMs and software development
Do you also disagree with the use of the name “tree” in a computer science class?

Again, nobody thinks trees in computer science contains squirrels, nobody thinks airplanes are birds, and nobody thinks a neuron in a ML model contains axons and dendrites. This is a weird hill to die on.

Are you gonna complain that the word “photograph” is “light writing” but in reality nobody is writing anything so therefore the word is wrong?

jychang··on Some thoughts on LLMs and software development
1. They are neurons, whether you like it or not. A binary tree may not have squirrels living in them, but it's still a tree, even though the word "tree" here is defined differently than from biology. Or are you going to say a binary tree is not a tree?

2. You are about 5 years behind in terms of the research. Look into hierarchical feature representation and how MLP neurons work. (Or even in older CNNs and RNNs etc). And I'm willingly using the word "neuron" instead of "feature" here because while I know "feature" is more correct in general, there are definitely small toy models where you can pinpoint an individual neuron to represent a feature such as a face.

jychang··on Some thoughts on LLMs and software development
I dislike the term “stochastic parrot”, because there’s plenty of evidence that LLMs do have an understanding of at least some things that they are saying.

We can trace which neurons activate for a face recognition model and see that a certain neuron does light up when it sees a face. The correct features are active for the sentence “the word parrots is plural”.

If you stop assuming the LLMs have no internal representations of the data, then everything makes a lot more sense! The LLM is FORCED to answer questions… just like a high school student filling out the SAT is forced to answer questions.

If a high school student fills out the wrong answer on the SAT, is that a hallucination?

Hallucinations are expected behaviors if you RLHF a high schooler to always guess an answer on the SAT because that’ll get them the highest score. This applies to ML model reward functions as well.

jychang··on Lucky 13: a look at Debian trixie
The biggest problem with Debian 13 is not with Debian, it's with people like Google and Cloudflare.

Come on guys, Debian 13 has been in testing for months, and you can't be arsed to update your apt repos from bookworm to trixie by release, or even weeks after release? That's embarrassing.

      ~ sudo apt update --audit
    [...]
    Hit:8 https://packages.cloud.google.com/apt google-compute-engine-bookworm-stable InRelease
    Hit:10 https://packages.cloud.google.com/apt cloud-sdk-bookworm InRelease
    Hit:11 https://pkg.cloudflareclient.com bookworm InRelease
    Hit:12 https://pkg.cloudflare.com/cloudflared bookworm InRelease
    [...]
    Fetched 407 kB in 2s (222 kB/s)
    2 packages can be upgraded. Run 'apt list --upgradable' to see them.
    Warning: https://pkg.cloudflare.com/cloudflared/dists/any/InRelease: Policy will reject signature within a year, see --audit for details
    Audit: https://pkg.cloudflare.com/cloudflared/dists/any/InRelease: Sub-process /usr/bin/sqv returned an error code (1), error message is:
       Signing key on FBA8C0EE63617C5EED695C43254B391D8CACCBF8 is not bound:
                  No binding signature at time 2025-08-21T15:58:52Z
         because: Policy rejected non-revocation signature (PositiveCertification) requiring second pre-image resistance
         because: SHA1 is not considered secure since 2026-02-01T00:00:00Z
These apt repos are still bookworm-only after the trixie release, and it's been weeks. And Cloudflare is still stuck on SHA1.
jychang··on Ask HN: The government of my country blocked VPN access. What should I use?
That hasn't worked for china since before 2020, you're years out of date.
jychang··on Asahi Linux Lead Developer Steps Down
> long-time Assahi Linux leader

Well, it's a shit edited article, but at least it's not AI shit

jychang··on Undisclosed financial conflicts of interest in DSM-5 (2024)
That sounds like not the psychologist’s fault. Don’t blame the doctor if their patient makes a sprained ankle their identity.

If someone smokes a lot of cigarettes and gets lung cancer, we don’t blame the doctor for the lung cancer. This shit is the patient’s responsibility.

jychang··on Ollama and gguf
Well llama.cpp requires minimum CUDA 11 from 2020, or if you need CUDA C++17 support, then CUDA 12 from 2022.

It'll compile fine against latest CUDA 12.8 or 12.9 etc, but there's zero need to pack whatever the latest CUDA version is.

jychang··on What's the strongest AI model you can train on a laptop in five minutes?
> here's never a size where the model cannot recognize faces at all

True

> then you add just a single neuron that recognizes them perfectly

Not true.

Don't think in terms of neurons, think in terms of features. A feature can be spread out over multiple neurons (polysemanticity), I just use a single neuron as a simplified example. But if those multiple neurons perfectly describe the feature, then all of them are important to describe the feature.

The Universal Approximation Theorem implies that a large enough network to perfectly achieve that goal would exist (let's call it size n or larger), so eventually you'd get what you want between 0 and n neurons.

jychang··on Linux address space isolation revived after lowering performance hit
That’d break a lot of GPU setups
jychang··on What's the strongest AI model you can train on a laptop in five minutes?
Why not? It takes models of a certain size to contain xyz neuron/feature.

https://www.youtube.com/watch?v=AgkfIQ4IGaM

That's not a mirage, it's clearly capability that a smaller model cannot demonstrate. A model with less parameters and less hidden layers cannot have a neuron that lights up when it detects a face.

jychang··on Ollama and gguf
?

    llamacpp> ls -l \*llama\*
    -rwxr-xr-x 1 root root 2505480 Aug  7 05:06 libllama.so
    -rwxr-xr-x 1 root root 5092024 Aug  7 05:23 llama-server
That's a terrible excuse, Llama.cpp is just 7.5 megabytes. You can easily ship a couple copies of that. The current ollama for windows download is 700MB.

I don't buy it. They're not willing to make an 700MB download a few megabytes bigger to ~730MB, but they are willing to support a fork/rewrite indefinitely (and the fork is outside of their core competency, as seen by the current issues)? What kind of decisionmaking is that?

jychang··on Ollama and gguf
That's a dumb answer from them.

What's wrong with using an older well-tested build of llama.cpp, instead of reinventing the wheel? Like every linux distro ever who's ever ran into this issue?

Red Hat doesn't ship the latest build of the linux kernel to production. And Red Hat didn't reinvent the linux kernel for shits and giggles.

jychang··on The Framework Desktop is a beast
For inference, Nvidia/AMD/Intel/Apple are all generally on the same tier now.

There's a post on github of a madman who got llama.cpp generating tokens for an AI model that's running on an Intel Arc, Nvidia 3090, and AMD gpu at the same time. https://github.com/ggml-org/llama.cpp/pull/5321

jychang··on The Framework Desktop is a beast
No, that's not true. Prompt processing just needs attention tensors in VRAM, the MLP weights aren't needed for the heavy calculations that a GPU speeds up. (After attention, you only need to pass the activations from GPU to system RAM, which is about ~40KB so you're not very limited here).

That's pretty small.

Even Deepseek R1 0528 685b only has like ~16GB of attention weights. Kimi K2 with 1T parameters has 6168951472 attention params, which means ~12GB.

It's pretty easy to do prompt processing for massive models like Deepseek R1, Kimi K2, or Qwen 3 235b with only a single Nvidia 3090 gpu. Just do --n-cpu-moe 99 in llama.cpp or something similar.

jychang··on The Framework Desktop is a beast
They're the same price. Both 395 machines (with 128GB of RAM) are $1999.
jychang··on The Framework Desktop is a beast
You're most likely bottlenecked by memory bandwidth for a LLM.

The AMD AI MAX 395+ gives you 256GB/sec. The M4 gives you 120GB/s, and the M4 Pro gives you 273GB/s. The M4 Max: 410GB/s (14‑core CPU/32‑core GPU) or 546GB/s (16‑core CPU/40‑core GPU).

jychang··on The Framework Desktop is a beast
AMD Ryzen AI Max 395+ as the APU (CPU + GPU), and its corresponding soldered LPDDR5X RAM, which has a super fast memory bandwidth of 256GB/sec.

So the CPU and RAM are soldered on the motherboard.

You can buy just the motherboard from Framework: https://frame.work/products/framework-desktop-mainboard-amd-...

jychang··on Abusing Entra OAuth for fun and access to internal Microsoft applications
To be fair, I’m pretty sure the code here was written before modern AI was a thing, back when dinosaurs roamed the earth.
jychang··on I want everything local – Building my offline AI workspace
... I mean, yes? The entire point of local AI is so you can feed your enterprise code into it, that you don't want offloaded to somewhere else.

That's the exact perfect use case for Docker, versus something heavier weight like a VM. What, you expect generic enterprise code to somehow be too dangerous for Docker but acceptable in a VM?

jychang··on I want everything local – Building my offline AI workspace
This is annoyingly Apple-only though. Even though my main dev machine is a Macbook, this would be a LOT more useful if it was a Docker container.

I'd still take a Docker container over an Apple container, because even though docker is not VM-level-secure, it's good enough for running local AI generated code. You don't need DEFCON Las Vegas levels of security for that.

And also because Docker runs on my windows gaming machine with a fast GPU with WSL ubuntu, and my linux VPS in the cloud running my website, etc etc. And most people have already memorized all the basic Docker commands.

This would be a LOT better if it was just a single docker command we can copy paste, run it a few times, and then delete if necessary.

jychang··on I want everything local – Building my offline AI workspace
Any way to install this via just a container?

Similar to a `docker compose up -d` that a lot of projects offer. Just download the docker-compose.yml file into a folder, run the command, and you're running. If you want to delete everything, just `docker compose down` and delete the folder, and the container and everything is gone.

Anything similar to that? I don't want to run a random install.sh on my machine that does god knows what.

jychang··on Why there are so many more South Asian CEOs than East Asian CEOs in the US
Honestly, this is probably the easiest and most accurate answer. India being a british colony for a long time means that the top 1% of Indians had a lot more exposure to English than the top 1% of Chinese people.

From there, statistics comes into play. There's just a bigger pool of Indians to select CEOs from.

jychang··on Ollama Turbo
They implement more openai-compatible endpoints than ollama at least
jychang··on Ollama Turbo
That’s just llama-swap and llama.cpp
jychang··on Perplexity is using stealth, undeclared crawlers to evade no-crawl directives
Who cares what Hacker News wants? You’re not obliged to participate in discussion.

Am I supposed to spend money on Amazon.com when I visit the website just because Amazon wants me to?

jychang··on Telo MT1
I fail to see how prejudice against waste is a problem.

Prejudice is a bad thing- for things that people can't change, like their race or age. Prejudice against people making bad or wasteful decisions is a good thing.

jychang··on M8.7 earthquake in Western Pacific, tsunami warning issued
https://www.google.com/maps/place/50°40'00.0"N+156°07'00.0"E...

That port right next to the water has probably disappeared.

← PreviousPage 9 of 28Next →