I'm shocked that interviewing still works how it does in large companies; the Sybil attacks and DDOS are just getting started.
926 karma · joined December 6, 2020
I'm shocked that interviewing still works how it does in large companies; the Sybil attacks and DDOS are just getting started.
We have a few whole-team meetups in Hong Kong each year for 2-4 weeks, and there's a San Diego or Hong Kong office that anyone can work from as they choose. We also have a wide array of fancy multi GPU boxes that everyone on the team gets full access to (known external contributors can get some access also).
I think many companies that were quick to embrace remote have walked it back, not everyone is capable of working productively remotely, nor are all types of work amenable to remote.
1. They built the agent and it's somehow competitive. If so, they shouldn't just replace their own job with it, they should replace a lot more jobs and get a lot more rich than one salary.
2. They rent the agent. If so, why would the renting company not rent directly to their boss, maybe even at a business premium?
I see no scenario where there's an "agent to do my work while I keep getting a paycheck."
There's a lot of scammers in the world, but OpenAI, Tesla, Amazon, and Microsoft have mostly made my life better. It's not about having money, look at all the startups that have raised billions and gone kaput. Vs say Amazon who raised just $9M before their $54M IPO and is still around today bringing tons of stuff to my door.
tinygrad doesn't support 3 yet, it's not needed on any AMD GPUs, and not needed on NVIDIA consumer. It wouldn't be hard to add, but it's important to figure out how it best fits with the existing abstractions. I think everything will eventually move to a more producer-consumer model.
Things are turning around for AMD. If you have an AMD card, go to pytorch.org, click Linux+ROCm and install PyTorch. 3 years ago, this was hopeless. Today, most mainline things work. I ran nanochat on MI300X and it just worked. I think that's true about MI350X now too. The MI350X machine is stable.
They are clearly behind NVIDIA, nobody doubts that. And a lot of investment into software will be required to catch up, ecosystem, compiler, and driver. But 2 years ago they seemed hopeless, now they don't. Things take time. HipKittens is a great codebase to study to see where AMD's LLVM backend is still lacking; compare it to the CUDA Kittens.
For training, it's NVIDIA and Google in first. AMD in second. And nobody in third. Intel and Tenstorrent are not remotely close. Huawei examples segfaulted. Groq gave up selling chips. Cerebras isn't available anywhere. Trainium had a 5 day wait time to get one instance and I lost interest.
It's not a "compiler," it's a "probabilistic code synthesizer guided by your constraints"
The latter is technically more specific and correct than the former, but it's 7 words instead of 1. And the word compiler is understood to encompass the latter, even if most compilers aren't that. They are both "a tool in a workflow"
"LLM coding tools are search-based program synthesizers," in my mind this is what compilers are. I think most compilers do far too little search and opt for heuristics instead, often because they don't have an integrated runtime environment, but it's the same idea.
"Plenty of effective engineering tools are stochastic," sure but while a SAT solver might use randomness and that might adjust your time to solve, it doesn't change the correctness of the result. And for something like a fuzzer, that's a test, which are always more of a best effort thing. I haven't seen a fuzzer deployed in prod.
"Determinism comes from external specs and tests," my dream is a language where I can specify what it does instead of how it does it. Like the concept of Halide's schedule but more generic. The computer can spend its time figuring out the how. And I think this is the kind of tools AI will deliver. Maybe it'll be with LLMs, maybe it'll be something else, but the key is that you need a fairly rigorous spec and that spec itself is the programming. The spec can even be constraint based instead of needing to specify all behavior.
I'm not at all against AI, and if you are using it at a level described in this post, like a tool, aware of its strengths and limitations, I think it can be a great addition to a workflow. I'm against the idea that it's a magical English compiler, which is what I see in public discourse.
Price discrimination is when two people visit a site to buy a book, the algorithm computes an estimate of what they are barely willing to pay, and then shows the two of them different prices for the exact same book based on who they are.
I know you think everyone is just trying to "get their bag" and that's the framework you see this in. But I already had more money at 21 than I've spent to date, and not cause I had a lot of money, but cause I don't buy much stuff. I'm sorry you feel played, but don't project that on me.
Like it's strange people think I give up on things, I think they listen to the media too much. This is a 2+ year long project that I've worked on almost every day. https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t...
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.3
today on stream and it just worked. No external ROCm install, and just the amdgpu driver that's in Arch.
We also have our own complete driver/runtime now in tinygrad; it's so much nicer to build on a foundation when you can blame yourself for the bugs.
That's my whole argument for gold. The finite supply isn't socially constructed, and getting more requires building real infrastructure.
Those drivers are faster than anything else when used to run fixed command queues (what neural network runs are)
We wrote entire NVIDIA, AMD, and QCOM drivers in that style.
https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...
https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...
https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...
Say you are training a 3B parameter model in BF16. That's 6GB of weights, as long as your step time is >=500ms you won't see a slowdown.
The symmetric half of the metldr key was obtained with a novel exploit I found. Then the asymmetric half was derived from that with the fail0verflow method.
But who really still cares about any of this, it was 13 years ago.