calebkaiser

It's a totally reasonable question, and one that everyone asks when they're learning about CUDA. The frustrating answer to your last question is that lots of companies have shipped GPU dev environments that can theoretically be used instead of CUDA. AMD has ROCm, Apple has had a couple projects (OpenCL, Metal), Intel has some stuff, and there are newer efforts like TinyGrad + a generation of slightly higher level frameworks from AI companies, like Triton from OpenAI.

The basic problem is that CUDA has become something of a Schelling point. If you want to train a model right now, the highest performance you can get is almost certainly on CUDA. From the basic general matrix multiply operation, to specific NN architectures, CUDA is going to have incredibly optimized implementations out of the box. And it's going to make multi-GPU training so much easier. And all the dependencies you build on (those layers you import from PyTorch or Transformers or whatever) are going to work optimally right away on CUDA. And that weird random repo that you found with a unique optimizer--it runs on CUDA too. And now the cool new implementation that you're about to release is also going to be built for CUDA.

It's so tempting to think "Just write replacement software", but you also need to transition the entire ecosystem in large part to match CUDA's effectiveness, and you need to get comparable performance out of your chip/library combo as NVIDIA can get out of its cards with CUDA.

There's a whole story here to how effective NVIDIA has been at navigating this. Very early on, they heavily prioritized PyTorch and TensorFlow, getting involved in the projects as much as they could and making sure they always ran best on CUDA. But the TLDR is that yes, you're right, another company could write a CUDA competitor. But actually replacing CUDA is a much larger task.

I'm personally hopeful that with the rise of coding agents, we see more movement on this front with other projects moving into view. It will take some time for any ecosystem to start to emerge that can dislodge CUDA for researchers who don't want to dive that deep into the stack, but hopefully we start to see some momentum build.

So OpenAI would be proving to the government that they should be trusted to govern models, by failing to govern their models?

Plenty of shady stuff goes on in marketing, and I'm sure that OpenAI is going to opportunistically grab any potential upside from this (and any other situation, generally). But it is hard to imagine that this is part of a premeditated master plan that went something like:

- Advocate that open models are ungovernable and that frontier labs should be trusted to safeguard autonomous agents

- Immediately fail to safeguard their autonomous agent

- Intentionally hack the company who is most publicly critical of their view, and who happens to be the center of the open model universe

- Collaborate with said company on reports that show that open models were in fact critical to mitigating their rogue agent

- So tightly control access to this plan at their 8,000 employee company that it never leaks

All in the hopes of generally getting the attention of the government, who would then hopefully (and inexplicably, given the details here) decide to give OpenAI more power?

There are probably easier ways to lobby politicians.

I think the commenter is saying that OpenAI committed a false flag operation, not Tailscale.

For Tailscale, this may very well be marketing, but it would be strangely self defeating for OpenAI to do something like what is being suggested. Showing that you failed to govern your model is a pretty poor way to say "We're the only ones who should be trusted to govern frontier models".

I'm typically very skeptical of most content marketing/corporate PR.

In the case of this incident, I struggle to see the clear upshot for OpenAI. It seems pretty unlikely they'd ever okay this intentionally as some sort of marketing.

For one, it'd be pretty damning when it leaked that this was a setup, and it would 100% leak at some point. But more importantly, it really flies in the face of the general argument frontier labs have been putting forth around the dangers of "ungovernable" open models and the role of frontier labs as responsible custodians. Members of OpenAI's leadership team were actually in the middle of a Twitter spat with HuggingFace employees/open model advocates about open models being generally decel and bad when this happened.

HF immediately got to show that they were only able to respond to the incident because of open models, that we can't rely on labs to be our sole source of stewardship, etc as a result of this.

Implementing models directly from papers is typically pretty doable (and is of course more straightforward when the full implementation is open sourced). Often there is some amount of specific knowledge, like particular hyperparameters, that is missing and has to be trial and errored by the community, but generally speaking, getting the core model architecture implemented is a reasonable task for most well documented models.

Reproducing the exact training run, however, is basically impossible without the original dataset and training pipeline (here meaning all of the code + infra involved in actually executing the pre and post training loops). Also, it would be exorbitantly expensive to do if you weren't also a lab trying to train a similar model.

But you can still scale the architecture down and experiment as a solo researcher using the published research. There are probably some open source implementations already on GitHub for any given big open model release.

OpenAI's head of strategic futures publicly stated that you can't explain the quality of the newest Kimi via distillation.

Further, you can just read the papers released alongside most open models. Plenty of hugely influential research results published that drive the frontier forward. It's not like these models are just existing architectures downloaded from Huggingface and trained on frontier lab APIs.

Here's his original post https://x.com/i/status/2078133895766114412

The guy who said those things just joined OpenAI in the last month. He was previously a senior policy advisor, specifically on AI, to the current administration in the White House.

I don't know that the US government has a unified position on open source AI. But it's not like this isn't a discussion happening among policymakers.

The conversation going on in the industry is a bit broader than that. OpenAI's head of strategic futures just said last week that open models are inherently decelerationist, ungovernable, and will slow development on the frontier. You'll notice OpenAI does not appear as a signatory here despite their relationship with Microsoft.

Money is a big part of it. The notable thing about Chinese open models is their size. Most open source models out of US labs have not tried to compete with the enormous flagship models you see from frontier labs now. Cohere, Arcee, Poolside, etc all put out open source models, and of course, Google has their Gemma line, Meta has their open models, OpenAI released an open weight 120b model, etc. The incentives are just tricky for running a massive training run for a fully open model. Monetizing can be hard, as if you want to monetize inference, you'll be competing against platforms like Fireworks etc that will also be serving your model.

Thinking Machines just released an enormous open model though, and I'm hopeful to see more of this globally. I believe open models are emphatically a good thing. But I do sympathize with smaller labs trying to figure out the economics.

I'm sympathetic to this POV. I think specifically in the case of Slack, it's not their responsibility in a cosmic sense, but it is in line with the direction and general promise they've been making to customers since forever.

A major part of Slack's go-to-market strategy for as long as I can remember has been pushing Slack as more than a interface for talking with teammates, but as a unified interface for generally piloting your business. Slack has heavily pushed ideas like "SlackOps", they've had Slackbot in the product forever, and they've always prioritized and sold integrations designed to let you build your business workflows in Slack. They even updated Slackbot to go from "that chatbot you DM to set reminders" to "Your AI teammate in Slack": https://slack.com/features/slackbot

So adapting to agents is in keeping with what Slack promises its users. I know that a lot of people don't want more AI features in apps that don't need them (i fall into this camp despite working in ML myself and using LLMs all day). I just think that in the case of Slack, their most passionate power-users are also the sort of people who do really want AI features.

Notion is probably a good companion here. It is similarly a really primitive utility--text editing--that has marketed itself as a sort of central interface for running your company. I know plenty of people, especially outside of engineering, for whom Notion is essentially their browser for work. And it's probably not a coincidence that the Notion + Slack are the two normal office software companies I can think of who've most heavily adopted and marketed AI features.

I am strongly in favor of open models, open source ML more broadly, and am pretty critical of the cynical positions adopted by major US labs vis a vis open models.

But this is an insane characterization. Literally every single researcher and executive at OpenAI and Anthropic would say that "these Chinese labs have a lot of talented people." They hire from them (and vice versa). Tencent's chief AI scientist was poached directly from Deepmind, who poached him from Anthropic, etc etc etc. Do you think there are just zero people from China working at US frontier labs?

And even beyond that, the entire ML ecosystem (including people at OpenAI and Anthropic) get excited about research published by Chinese labs. Deepseek's GRPO paper set the ecosystem on fire for a little while.

The contention from OpenAI and Anthropic around distillation has basically been "Labs that distill from us get to bootstrap their model at a much lower price point". Or, in other words, "If we didn't invest in building the teacher model, it wouldn't be possible for these labs to distill their student model." Which I'm not very sympathetic to, but is a far cry from how you're characterizing it.

OpenAI's Head of Strategic Futures just this week posted this about the latest Kimi release: "It's a very good model! I don't think its performance can be explained away by distillation or anything like that."

It was part of a longer post that kicked off quite a firestorm about open models and OpenAI's position on them, but it's also notable that labs are no longer contending that open models are essentially just distilled versions of frontier models: https://x.com/deanwball/status/2078133895766114412

This was my initial reaction to reading this post as well.

Additionally, as I get older, I find the sentiment of "we're not trying to build a perfect system here" is less about "let's just go fast vroooom" and more akin to saying "I've been humbled before by thinking I had the perfect mental model of the universe before a single user touched the product."

This is not my experience working in the field the last 8 years. There is not a dearth of talented researchers, engineers, etc. who are willing to contribute to open models. Just look at the ecosystem generally, from academia to industry. So much research still happens in the open, and the open source community in ML is still massive and energetic.

Training a new frontier model just costs a lot of money and there are very few companies for whom it makes sense to do so, and even fewer orgs who are just going to gift those kinds of resources to an open source initiative.

Gwern's absurdly catalogued personal site is one of those online artifacts that I hope never changes.

Based on a cursory read of the situation, it seems similar (at least on its face) to the Waymo vs Uber situation. In that case, Uber payed a Waymo an equity stake and signed an agreement about which technology they would/wouldn't use. The key person involved also was sentenced to 18 months in prison (pardoned after 6 months).

I mean, OpenAI delayed the public release of GPT-2 back in 2019 because it seemed capable of authoring interesting blog posts (that also happened to be untrue). It was a pretty big deal the first time Transformer models were capable of generating that kind of output--no one found it weird. We've just grown to take it for granted that large Transformer models are this capable.

The same cycle is happening now for a harder frontier. And proofs represent a pretty good benchmark for model capabilities, so a new model proving a result that a previous model didn't is generally notable in the same way that a model scoring higher on a benchmark is.

I'm sure we'll take it for granted in the not-too-distant future.

I think the author largely agrees with you re: type systems and LLMs. He's pretty explicit that Haskell should be very well positioned to be a power language for LLM-assisted programming, but that the Haskell ecosystem presents the bottlenecks that make it harder.

I don't personally use Haskell for anything, but I use Lean and occasionally some other languages with expressive type systems, and like you I've found it to be a pretty great experience for working with LLMs. But I've also experienced what the author is talking about, with languages that sit at different points on the type system spectrum, regarding a languages ecosystem/infra layer becoming a bottleneck. I don't think it's ultimately about the type system but the broader ergonomics of the language/ecosystem.

So I think his criticism is less than expressive type systems are a pre-LLM concept, and more that Haskell has an individually bad "agentic coding story".

I've been a power user of LLMs for software development for a while now, and I've found two things to be true:

- The benefits of more "extreme" type systems are more accessible and valuable than ever. I have a fairly involved project built on Lean that I hope to open source this month, and it's been a joy to work in even for uses outside of mathematics.

- Readability, build time, infra complexity, and everything that affects your speed after finishing your implementation--these things now matter more than ever.

It's sort of a dual ergonomics problem, in some sense. And given that, the author's lament makes complete sense to me, especially:

"An AI-enabled Haskell ecosystem would ask different questions. How do we make Haskell easier for agents to use well? How do we get more high-quality Haskell examples into model training data? How can we scale reviews? How do we make library docs full of copy-pastable, realistic examples, not just beautiful types? How do we make project bootstrap fast? How do we make error messages more agent-friendly? How do we reduce cold build times? How do we make common industrial patterns obvious to a model that is trying to help?"

Lots of researchers have done just this! There's a really rich history of research + lots of contemporary work on different encoding/representation strategies. This might be interesting to you: https://sbert.net/

What makes the DeepSeek-OCR and related results exciting to some researchers is less about the fact that you could devise a tokenization scheme that has fewer tokens, and more about how well it works.

Nah, optical compression is a thing. You see it in a lot of different areas in ML. In this case, the "trick" has been known for a while, and belongs to a whole world of compression research. But I think where you're maybe getting mixed up is in where that 60% gain is coming from.

It's not a 60% percent reduction in cost for 100% of the same output. If you have a model and input text A, and you fix the seed etc. and run Text A through the model as text tokens and as compressed image tokens, you will not get identical outputs. You're specifically reducing the number of tensors needed to represent your input, which saves you on raw compute, but also by definition gives you less room to represent the information in your input. It's lossy, in other words.

Put another way, if you're using a model like Fable because you need the absolute frontier of capability and cheaper models cannot solve your tasks, then there is a very real chance that a compression strategy like this drops Fable's accuracy such that it's no longer suitable for your task. Which defeats the point of you paying for the most expensive model in the first place.

So, it's cool research. Might be useful for some people. Probably isn't something that has incredible utility in real use cases.

This has been a (noble) goal of lots of different projects in the community for a long time. Federated learning projects like Flower have been chipping away at it for a long time. There are many many hurdles to be cleared before anything in this area is super feasible as an alternative, but I applaud everyone who works on it.

This is a good starting point: https://huggingface.co/docs/peft/developer_guides/model_merg...

But yes, in general, merging refers to techniques that directly blend the weights of different models mathematically. It had a big moment of popularity ~2 years ago, with many so-called "Frankenmodels" popping up on leaderboards.

I tend to think of merging as belonging to the same general umbrella as things like "abliteration", or other techniques that surgically modify the weights of a model without a traditional training/tuning loop. Maxime Labonne is a great person to follow if you're interested in this general area.

I don't understand this line of criticism exactly. By putting new information in the context window, you are materially changing the activations at your point of sampling, which is literally "customizing with mere markdown files."

Taken to the extreme, the attitude that there is some special incantation that will unlock all capabilities is silly, and a lot of the "prompt engineering" discourse is similarly kind of dumb, but in-context learning is clearly a real thing.

Eh, Watson was a classic open domain QA system originally, no deep learning or much of what we think of in an "AI platform" today. It was one of a bunch of such systems that were built in that early 2000s period. They all failed because the approach fundamentally didn't work very well.

Here's a write up of some relevant history if you're curious https://liweinlp.com/1465

My experience has been that this is not unique to tech, and is common in all large enough industries. I think it's just the natural emergence of reward hacking i.e. if you're an executive at Pepsi and your job is largely to increase the stock price, and you know that you can do something to change the way your numbers are presented such that Wall St will like it, you'll likely do it.

I do think tech certainly has its own flavor though, particularly because of how differently it is treated by investors.

If I'm remembering right, it was weirder than that, as Llama's originally release strategy was sort of bizarre.

You did have to apply for access, but if you met their criteria (basically if you were the right profile of researcher or in government), you got direct access to the model weights, not just an API for a hosted model. So access was restricted, but the full weights were shared.

I believe that the model was leaked by multiple people, some of which didn't work at Meta but had been granted access to the weights.

2 years? 2 years ago, gpt-4o was OpenAI's flagship model. The gap is real, but much smaller than 2 years.

I've worked on two open source infrastructure projects that raised money now, and am friends with people involved in many more. I'd put a couple of asterisks next to the claims in this article:

- VCs definitely cared about our Stars, especially in early stages, but not as our primary metric. I suppose Stars might be the primary metric if they're truly off the charts, but usually they're just one of many social proof signals an investor might look at.

- Investors, especially at the earliest stages, are quite a varied bunch. Some were diligent about looking at who was leaving Stars on the repo (i.e. are these accounts fake/do they belong to potential future customers). Some less so. This is true for basically every metric (see: startups that grossly misreport ARR)

- Fake GitHub stars were a thing way before 2022. I'd have to look in more detail at the methodology here, but I'd question any analysis that finds that paying for GitHub Stars (or any social following kind of metric) is a strictly post-2022 thing. Any metric that can be construed as social proof will immediately have its own grifter economy. Investors know this and (mostly) do their diligence.

Finally, showing numbers is hard for an early stage open source startup. At later stages, you should be able to show an actual business with typical metrics, but at the seed stage you often just have a repo and a website. Your goal is just to get a lot of people using your software. You can add telemetry to track that, but that's a thorny decision. GitHub Stars aren't a terrible proxy for popularity, provided that you audit the quality of the following. A project with a lot of organic stars and forks is, at the very least, a project that a lot of people are familiar with.

I'm not saying that GitHub Stars aren't wildly overvalued or gamed, but contextualized properly, they're a reasonable metric to consider, particularly at earlier stages. Most investors aren't just throwing millions at random repositories with 20k Stars from obviously spam accounts.