HNHacker News
TopNewBestAskShowJobs

rahen

1,563 karma · joined May 25, 2021

Systems engineer making Lisp great again for machine learning: https://github.com/sheaf-lang/sheaf
submissionscomments
rahen··on Greece to ban anonymity on social media
I wonder how they plan to implement that on decentralized social networks such as Nostr. I assume targeting big centralized networks such as X and Facebook is good enough.
rahen··on Show HN: How LLMs Work – Interactive visual guide based on Karpathy's lecture
Some subreddits have already taken measures against this: https://www.reddit.com/r/ProgrammingLanguages/comments/1sd66...

Same everywhere: avalanches of AI garbage and intellectual dishonesty. People claiming "I wrote this", then a look at the code shows massive slop and an author with no clue about the topic.

More worrying, this trend is creeping to all domains: "Nearly 75,000 tracks uploaded to Deezer are fully created using AI. That’s 44% of daily uploads, and more than 2 million per month. Back in June, the daily number was around 20,000."

https://www.vice.com/en/article/how-deezer-is-fighting-fraud...

rahen··on Windows 9x Subsystem for Linux
Before WSL, the best ways to run unmodified Linux binaries inside Windows were CoLinux and flinux.

http://www.colinux.org/

https://github.com/wishstudio/flinux

flinux essentially had the architecture of WSL1, while CoLinux was more like WSL2 with a Linux kernel side-loaded.

Cygwin was technically the correct approach: native POSIX binaries on Windows rather than hacking in some foreign Linux plumbing. Since it was merely a lightweight DLL to link to (or a bunch of them), it also kept the cruft low without messing with ring 0.

However, it lacked the convenience of a CLI package manager back then, and I remember being hooked on CoLinux when I had to work on Windows.

rahen··on Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
Yes. The author mentions Claude for testing, but it was obviously used for the README and code as well.

This is a giveaway for AI generation, from the docstring to the terrible opcode dispatch (Claude sucks at assembly or low-level optimization): https://github.com/gizmo64k/soulplayer-c64/blob/main/src/cpu...

A human would use a proper dispatch table and wouldn't make excuses for a sloppy implementation ("Python is fast enough").

Besides, the author has an art and design background, which doesn't seem to match the deep knowledge of Transformers or assembly required for such a project.

rahen··on Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
A little disappointed to see PyTorch + Claude here. I was hoping for some "demo-scene" hand-crafted 6502 assembly, and hopefully training on the C64.
rahen··on Want to Write a Compiler? Just Read These Two Papers (2008)
I'm also writing a compiler and CS6120 from Cornell has helped me a lot: https://www.cs.cornell.edu/courses/cs6120/2025fa/self-guided...
rahen··on Show HN: I built a tiny LLM to demystify how language models work
I don't mean to be 'that guy', but after a quick review, this really feels like low-effort AI slop to me.

There is nothing wrong using AI tools to write code, but nothing here seems to have taken more than a generic 'write me a small LLM in PyTorch' prompt, or any specific human understanding.

The bar for what constitutes an engineering feat on HN seems to have shifted significantly.

rahen··on Circuit-level PDP-11/34 emulator
Out of curiosity, what were those wonderful things you were hearing about the 11/34 back then?
rahen··on Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer
Yes. The Cray supercomputers from the 80s were crazy good matmul machines in particular. The quad-CPU Cray X-MP (1984) could sustain 800 MFLOPS to 1 GFLOPS, and with a 1 GB SSD, had enough computer power and bandwidth to train a 7-10M-parameter language model in about six months, and infer at 18-25 tok/sec.

A mid-90s Cray T3E could have handled GPT-2 124M, 24 years before OpenAI.

I also had a punch-card computer from 1965 learn XOR with backpropagation.

The hardware was never the bottleneck, the ideas were.

rahen··on Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer
I also have a working design for a small Transformer on the original Game Boy. It has around 4000 parameters fitting in the 8 KB cartridge SRAM, where the "saved game" is the trained model. A TI-82 with its 32 KB of RAM would be even more comfortable.
rahen··on Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer
I encouter two main failure modes. First, the bipolar PROMs degrade at the atomic level, the metal ions in the fuses tend to migrate or 'regrow' over decades, causing bit rot. Second, the backplanes suffer from mechanical fatigue. After forty years of thermal expansion and structural flexing, especially when inserting boards, the traces and solder joints develop stress cracks. Both are a pain to repair.

https://retrocmp.com/articles/trying-to-fix-a-dec-pdp-1134-b...

rahen··on Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer
The WASM GUI is probably the easiest way to see the Transformer in action on this machine: https://dbrll.github.io/ll-34/

There's also the original Tetris from 1984 to play.

rahen··on Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer
That thing is a Tamagochi though, it constantly needs attention, pardon the pun. I did most of the development and tuning on ll-34 for that reason.
rahen··on Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer
Thanks for reposting! I'm the author of ATTN-11. Happy to answer any questions about the fixed-point arithmetic, the PDP-11 hardware, or the training process.
rahen··on The Connection Machine CM-1 "Feynman" T-shirt
Around the same time (1984), there was also another very cool piece of technology that often gets overlooked: the CMU WARP. It wasn’t as flashy as the Crays and the Connection Machine, but it was the first systolic array accelerator (what we’d now call TPUs). It packed as much MFLOPS as a Cray 1.

It's also the computer that powered the Chevrolet Navlab self-driving car in 1986.

rahen··on Sheaf – A Functional Language for Differentiable Programs
I've been building a functional language for differentiable programming that compiles to JAX. The core idea is homoiconicity applied to ML, models are data structures that can inspect and transform themselves.
rahen··on The Emacs Widget Library: A Critique and Case Study
For those interested, this guy is revamping the Emacs widget library with something more modern and platform agnostic, based on SDL: https://appetrosyan.github.io/posts/

His posts are very insightful.

rahen··on I switched from VSCode to Zed
My only complaint regarding the Zed editor is the inability to display two panes of the sidebar one below the other. Not only is it impossible to display them together, but switching between them requires clicking a tiny button in the status bar. To make matters worse, performing a search hides the symbols and the tree view.

So right now I'm sticking to Emacs.

rahen··on Go away Python
Lisp isn't missing anything, it's a natural fit for AI/ML. It’s the ecosystem's tooling that needs catching up.

The code hasn't reached RC yet, but I'll definitely post a Show HN once it's ready for a preview.

rahen··on Go away Python
I doubt it considering there are massive Clojure codebases with large teams collaborating on them every day. The lack of Lisp tooling and the prevalence of Python are more a result of inertia, low barrier to entry and ecosystem lock-in.
rahen··on Go away Python
"> I think all of ML being in Python is a colossal mistake that we'll pay for for years.

Market pressure. Early ML frameworks were in Lisp, then eventually Lua with Torch, but demand dictated the choice of Python because "it's simple" even if the result is cobbled together.

Lisp is arguably still the most suitable language for neural networks for a lot of reasons beyond the scope of this post, but the tooling is missing. I’m developing such a framework right now, though I have no illusions that many will adopt it. Python may not be elegant or efficient, but it's simple, and that's what people want.

rahen··on Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
I love it, instant Github star. I wrote an MLP in Fortran IV for a punched card machine from the sixties (https://github.com/dbrll/Xortran), so this really speaks to me.

The interaction is surprisingly good despite the lack of attention mechanism and the limitation of the "context" to trigrams from the last sentence.

This could have worked on 60s-era hardware and would have completely changed the world (and science fiction) back then. Great job.

rahen··on Learning Fortran (2024)
The reverse is true though, and I find that fascinating with Fortran.

I recently learned Fortran IV to build a backpropagated neural network for the IBM 1130 (1965) and was amazed to see it compile with no warning on both the punched card compiler from IBM and on a modern Fortran compiler (gfortran).

Some Fortran II conventions, like the arithmetic IF, are now deprecated, but --std=legacy is all it takes to make it work.

rahen··on Eurydice: a Rust to C compiler
You mean everywhere. It's just hidden behind abstraction layers or Fortran libraries like BLAS/LAPACK, which are used by NumPy, R, Julia, MATLAB, Excel, TensorFlow, PyTorch (for some backends), and basically anything that involves linear algebra.
rahen··on You want microservices, but do you need them?
Unless you need horizontal scalability or clustering, Compose + Terraform is all you need.

With Compose, you get proper n-tier application containerization with immutability. By adding an infrastructure-as-code tool such as Terraform to abstract your IT environment, you can deploy your application on-premises, in the cloud, or at a customer site with a single command.

For clustering needs, there’s Incus, and finally, Kubernetes for very fast scalability (<mn), massive deployments on large clusters, cloud offloading, and microservices.

Almost nobody truly needs the complexity of Kubernetes. The ROI simply isn’t there for the majority of use cases.

rahen··on Making Crash Bandicoot (2011)
Part 9 elaborates on GOOL, the Lisp dialect they designed in-house to create the gameplay.

This is my favorite part: https://all-things-andy-gavin.com/2011/03/12/making-crash-ba...

rahen··on Runit Linux: Complete Guide to Unix Init Scheme with Service Supervision
The best challenger to systemd in terms of feature parity is probably dinit: https://davmac.org/projects/dinit/

Have a look at Chimera Linux if you want to give it a try: https://chimera-linux.org/

runit, s6, and OpenRC don't have the downsides of systemd, but they also only cover a subset of its features

rahen··on Runit Linux: Complete Guide to Unix Init Scheme with Service Supervision
The article and discussion are about runit, why bring systemd into it? Diversity in solutions is a good thing, there’s no need to feel threatened by that.
rahen··on Xortran - A PDP-11 Neural Network With Backpropagation in Fortran IV
Author here. They call it a FORTRAN IV compiler but it uses some F66 extensions, such as proper types and functions, although it lacks some of the nicer constructs of F66 like If/Then/Else, which would have been handy.

Regarding floating point, I realized the code actually works fine without an FPU, so I assume it uses soft-float. There's no switch to enable the FP11 opcodes, maybe that was in their F77 compiler.

It's indeed rough and spartan, but using a 64KB optimizing compiler requiring just 8KB of memory was a refreshing change for me.

rahen··on Xortran - A PDP-11 Neural Network With Backpropagation in Fortran IV
The first convolutional neural network, the Neocognitron, was AFAIK implemented on a PDP-11 as well: https://www.semanticscholar.org/paper/Neocognitron%3A-A-neur...

No backpropagation back then, this only appeared around 1986 with Rumelhart, probably on VAX machines by that time.

The 11/34 was hardly a powerhouse (roughly a turbo XT) but it was sturdy, could handle sustained load and its FPU made the whole difference.

← PreviousPage 2 of 19Next →