HNHacker News
TopNewBestAskShowJobs

georgehotz

926 karma · joined December 6, 2020

submissionscomments
georgehotz··on Five Years of Tinygrad
What's even sadder is that because of AI coding agents it's possible the bounty system will go away. The value today is much less in writing the code and much more in defining the tasks and validating the code. We don't have the bandwidth to read tons of AI slop, which at first glance looks okay, but upon spending time to read it you realize it isn't good. The fix for this is probably a reputation or staking system.

I'm shocked that interviewing still works how it does in large companies; the Sybil attacks and DDOS are just getting started.

georgehotz··on Five Years of Tinygrad
tinygrad has a PyTorch frontend, we fully encourage the frontend to stay PyTorch and for them to integrate tinygrad at any level. Our mission is to commoditize the petaflop, all frontends are welcome. We also have a quite complete ONNX frontend.
georgehotz··on Five Years of Tinygrad
That's comma.ai's policy since they make hardware and solve physical problems. The tiny corp has been hybrid (remote-first) since day 1 because it primarily writes open source software, and there's a long track record of success with remote for this kind of task.

We have a few whole-team meetups in Hong Kong each year for 2-4 weeks, and there's a San Diego or Hong Kong office that anyone can work from as they choose. We also have a wide array of fancy multi GPU boxes that everyone on the team gets full access to (known external contributors can get some access also).

I think many companies that were quick to embrace remote have walked it back, not everyone is capable of working productively remotely, nor are all types of work amenable to remote.

georgehotz··on Advent of Code 2025
I mean, everyone else was using them too, how can you not? That was the name of the game if you wanted to be competitive in 2024. Not using them would be like trying to do competitive pro cycling without steroids, basically impossible.
georgehotz··on Advent of Code 2025
I support the no global leaderboard. I was in 7th place last year but quickly got bored maintaining the aggressive AI pipeline required to achieve that. If I wanted to maintain pipelines I'd just do work, and there will never be a good way to prevent people from using AI like this. Advent of Code should be fun, thank you for continuing to do it. I'm looking forward to casually playing this year!
georgehotz··on AI CEO – Replace your boss before they replace you
comma.ai owns a lot of cars, including a Tesla, so I have tried most cars in the price range. Tesla is certainly no more of a scam than the other cars, and compared to say, the Chevy Bolt, it's a lot better. Can you suggest a better car for the value? Is there another car I can buy with better full self driving?
georgehotz··on AI CEO – Replace your boss before they replace you
Musk sells several things. Electric cars for $40k-$100k. Satellite internet for $40-$120 per month. X/Grok premium for $8/mo. And space launch services for about $2,500 per kg. Which one(s) of these are the scam? Prices seem decent to me, but if you tell me where I can get cheaper and better I'm open to it.
georgehotz··on AI CEO – Replace your boss before they replace you
Maybe a few people managed to outsource their own job and sit in the middle for a bit. But that's not the common story, the common story is that your employer cut out the middle man and outsourced all the jobs. The same thing will happen here.
georgehotz··on AI CEO – Replace your boss before they replace you
How do they own it? I see two scenarios.

1. They built the agent and it's somehow competitive. If so, they shouldn't just replace their own job with it, they should replace a lot more jobs and get a lot more rich than one salary.

2. They rent the agent. If so, why would the renting company not rent directly to their boss, maybe even at a business premium?

I see no scenario where there's an "agent to do my work while I keep getting a paycheck."

georgehotz··on AI CEO – Replace your boss before they replace you
Do people on Hacker News actually believe this? Each one of the four people named built a product I happily pay for! Then they used investment and profits to hire people to build more products and better products.

There's a lot of scammers in the world, but OpenAI, Tesla, Amazon, and Microsoft have mostly made my life better. It's not about having money, look at all the startups that have raised billions and gone kaput. Vs say Amazon who raised just $9M before their $54M IPO and is still around today bringing tons of stuff to my door.

georgehotz··on AI CEO – Replace your boss before they replace you
You are welcome to try to cut them out and start your own business. But I suspect you might find it a bit harder than your employer signing up for a SaaS AI agent. Actually wait, isn't that what this website is? Does it work?
georgehotz··on AI CEO – Replace your boss before they replace you
Who's giving you that paycheck? Why don't they just hire that AI agent themselves and cut out the middle man?
georgehotz··on HipKittens: Fast and furious AMD kernels
1 and 2 are supported, 1 you need to specify, 2 will be found with BEAM. We are working on reimplementing HipKittens in tinygrad, all the stuff is there to do it. See the amd_uop_matmul example.

tinygrad doesn't support 3 yet, it's not needed on any AMD GPUs, and not needed on NVIDIA consumer. It wouldn't be hard to add, but it's important to figure out how it best fits with the existing abstractions. I think everything will eventually move to a more producer-consumer model.

georgehotz··on HipKittens: Fast and furious AMD kernels
Nope! Works fine with in-tree somewhat recent kernel. The AMD driver is actually open source, not just a wrapper into a big on device blob like the NVIDIA one. tinygrad also has a driver that doesn't even need the kernel module, just mmapping the PCIe BAR into Python.
georgehotz··on HipKittens: Fast and furious AMD kernels
Full disclosure, we have a contract with AMD to get Llama 405B training on MI350X on MLPerf.

Things are turning around for AMD. If you have an AMD card, go to pytorch.org, click Linux+ROCm and install PyTorch. 3 years ago, this was hopeless. Today, most mainline things work. I ran nanochat on MI300X and it just worked. I think that's true about MI350X now too. The MI350X machine is stable.

They are clearly behind NVIDIA, nobody doubts that. And a lot of investment into software will be required to catch up, ecosystem, compiler, and driver. But 2 years ago they seemed hopeless, now they don't. Things take time. HipKittens is a great codebase to study to see where AMD's LLVM backend is still lacking; compare it to the CUDA Kittens.

For training, it's NVIDIA and Google in first. AMD in second. And nobody in third. Intel and Tenstorrent are not remotely close. Huawei examples segfaulted. Groq gave up selling chips. Cerebras isn't available anywhere. Trainium had a 5 day wait time to get one instance and I lost interest.

georgehotz··on AI coding
This is bait. The comment and the blog post say mostly the same thing, the debate is around the subtle edges.

It's not a "compiler," it's a "probabilistic code synthesizer guided by your constraints"

The latter is technically more specific and correct than the former, but it's 7 words instead of 1. And the word compiler is understood to encompass the latter, even if most compilers aren't that. They are both "a tool in a workflow"

georgehotz··on AI coding
Author here. I agree with this comment, but if I wrote more like this my blog post would get less traction.

"LLM coding tools are search-based program synthesizers," in my mind this is what compilers are. I think most compilers do far too little search and opt for heuristics instead, often because they don't have an integrated runtime environment, but it's the same idea.

"Plenty of effective engineering tools are stochastic," sure but while a SAT solver might use randomness and that might adjust your time to solve, it doesn't change the correctness of the result. And for something like a fuzzer, that's a test, which are always more of a best effort thing. I haven't seen a fuzzer deployed in prod.

"Determinism comes from external specs and tests," my dream is a language where I can specify what it does instead of how it does it. Like the concept of Halide's schedule but more generic. The computer can spend its time figuring out the how. And I think this is the kind of tools AI will deliver. Maybe it'll be with LLMs, maybe it'll be something else, but the key is that you need a fairly rigorous spec and that spec itself is the programming. The spec can even be constraint based instead of needing to specify all behavior.

I'm not at all against AI, and if you are using it at a level described in this post, like a tool, aware of its strengths and limitations, I think it can be a great addition to a workflow. I'm against the idea that it's a magical English compiler, which is what I see in public discourse.

georgehotz··on Are we the baddies?
That's not price discrimination, like hardcover vs paperback if you have two versions of something and people can choose which they want. That's totally fine and actually something that makes capitalism great. The rich usually end up covering more of the costs here cause they are less price sensitive, like business vs economy on airplanes.

Price discrimination is when two people visit a site to buy a book, the algorithm computes an estimate of what they are barely willing to pay, and then shows the two of them different prices for the exact same book based on who they are.

georgehotz··on Are we the baddies?
Nice try. I worked at Facebook for 9 months and left (before vesting any shares) because I didn't agree with the mission, even back in 2012. I worked at Twitter for 5 weeks and left because I realized nothing was going to change (and the good food went away). I don't regret trying at either, but in revealed preferences, I've spent most of my life writing open source software, even if that's not what attracts most media about me.

I know you think everyone is just trying to "get their bag" and that's the framework you see this in. But I already had more money at 21 than I've spent to date, and not cause I had a lot of money, but cause I don't buy much stuff. I'm sorry you feel played, but don't project that on me.

georgehotz··on Advice to Tenstorrent
We got the MI300X box on MLPerf too, and every MLPerf from here on general tinygrad improvements should bring down the times. We're still quite focused on AMD.

Like it's strange people think I give up on things, I think they listen to the media too much. This is a 2+ year long project that I've worked on almost every day. https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t...

georgehotz··on Advice to Tenstorrent
AMD has legitimately been making great progress. They still have a long way to go, and I appreciate SemiAnalysis taking up the mantle of calling them out, but I ran:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.3

today on stream and it just worked. No external ROCm install, and just the amdgpu driver that's in Arch.

We also have our own complete driver/runtime now in tinygrad; it's so much nicer to build on a foundation when you can blame yourself for the bugs.

georgehotz··on The Demoralization is just Beginning
Unlike gold, the "finite supply" of BTC is socially constructed.

That's my whole argument for gold. The finite supply isn't socially constructed, and getting more requires building real infrastructure.

georgehotz··on Tinygrad will be the next Linux and LLVM
FFI? This isn't how GPUs work...they are MMIO (mostly)

Those drivers are faster than anything else when used to run fixed command queues (what neural network runs are)

georgehotz··on Tinygrad will be the next Linux and LLVM
This sounds like prejudice. Have you benchmarked it?
georgehotz··on Tinygrad will be the next Linux and LLVM
Umm, why not?

We wrote entire NVIDIA, AMD, and QCOM drivers in that style.

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

georgehotz··on "Tinyboxes finally have a buy it now button"
Specs on website updated, anywhere from 100-240V is fine
georgehotz··on "Tinyboxes finally have a buy it now button"
That OCP 3.0 card has the same link bandwidth as the GPUs, so you can scale out without much loss of all-reduce bandwidth. In practice, for all models except the largest, the ~16GB/s all-reduce is totally fine. You just need to make sure you can all-reduce all weights in your training step time.

Say you are training a 3B parameter model in BF16. That's 6GB of weights, as long as your step time is >=500ms you won't see a slowdown.

georgehotz··on "Tinyboxes finally have a buy it now button"
It's the same way NVIDIA states bandwidth for PCIe and NVLink.

https://www.nvidia.com/en-us/data-center/h100/

georgehotz··on Comma.ai: Refactoring for Growth
The worst part of this critique is that it's not even true, and marcan knows better. He clearly dislikes me so much that he is willing to lie.

The symmetric half of the metldr key was obtained with a novel exploit I found. Then the asymmetric half was derived from that with the fail0verflow method.

But who really still cares about any of this, it was 13 years ago.

georgehotz··on Comma.ai: Refactoring for Growth
A GM at Ford absolutely will be paying this if they want a phone call. Joining our Discord is free however.
← PreviousPage 2 of 4Next →