Or, like Meta, they make their money elsewhere and just seem interested in wrecking the economics of LLMs. As soon as an open-weight model is released, it basically sets a global floor that says "Models with similar or worse performance effectively have zero value," and that floor has been rising incredibly quickly. I'd be surprised if the vast, vast majority of queries ChatGPT gets couldn't get equivalently good results from llama3/deepseek/qwen/mistral models, even for those paying for the pro versions.
It does seem like it will be very, very hard for the companies training their own models to recoup their investment when the capabilities of open-weight models catch up so quickly - general purpose LLMs just seem destined to be a cheap commodity.
The $500 going to the "AI Owner" instead of labor (i.e. the human lawyer) is the productivity gain though, right? And if that was such a productivity gain (i.e. the marginal cost was basically 0 to the AI owner, instead of, say, $499 in electricity and hardware), the usual outcome is that the cost for such a product/service basically gets driven to 0, and the benefit from productivity actually gets distributed to the clients that would have paid the lawyer (who suddenly get much cheaper legal services), rather than the owner of the 'AI lawyer.'
We seem pretty likely to be headed towards a future where AI-provided services have almost no value/pricing power, and just become super low margin businesses. Look at all of the nearly-identical 'frontier' LLMs right now, for a great example.
It's an HP Z8 G4 (dual-socket 18-core, 3 GHz Xeons, 24x32GB of DDR4-2666, and then a crappy GPU, 8TB HDD, 1TB SSD). It can accommodate 3 dual-slot GPUs, but I was mostly interested in playing with frontier models where holding all the weights in VRAM requires a ~$500k machine. It can run the full Deepseek R1, Llama3-405B, etc, usually around 1-2 tokens/sec.
I have a used workstation I got for $2k (with 768GB of RAM) - using the Q4 model, I can get about 1.5 tokens/sec and use very large contexts. It's pretty awesome to be able to run it at home.
I have a similar setup running at about 1.5 tokens/second, and it's perfectly usable for the sorts of difficult tasks one needs a frontier model like this for - give it a prompt and come back an hour or two later. You interact with it like e-mailing a coworker. If I need an answer back in seconds, it's probably not a very complicated question, and a much smaller model will do.
I can run the 671B-Q8 version of R1 with a big context on a used dual-socket Xeon I bought for about $2k with 768GB of RAM. It gets about 1-1.5 tokens/sec, which is fine to give it a prompt and just come back an hour or so later. To get to many 10s of tokens/sec, you would need >8 GPUs with 80GB of HBM each, and you're probably talking well north of $250k. For the price, the 'used workstation with a ton of DDR4' approach works amazingly well.
Being able to see the thinking trace in R1 is so useful, as you can go back and see if it's getting stuck, making a wrong assumption, missing data, etc. To me that makes it materially more useful than the OpenAI reasoning models, which seem impressive, but are much harder to inspect/debug.
"Packaging" in this context means taking the wafer of compute die (made in Arizona), dicing it up into individual die, mounting it onto a silicon interposer (an even bigger die, no idea where that's made, but probably taiwan) along with a bunch of HBM die, then mounting that Si interposer on a somewhat larger, very fine-pitched circuit board ('substrate') that is essentially a breakout for power and high-speed I/O from the compute die. That thing is the packaged 'CoWoS' system, where CoWoS==Chip-on-wafer-on-substrate, that eventually gets attached to a 'normal' PCB.
The word 'recession' means something though, and it's neutral on whether or not any particular humans have more income or cheaper goods. If you get a raise at work, it doesn't mean we're not in an economic recession as a country, and if you get fired or the price of gas goes up, it doesn't mean we are.
Block RAM in FPGAs is very limited and adding something like a DDR3 controller is a hassle - of the dev boards I have lying around, they have between 48KB and 200KB of block RAM. The actual CPU logic is probably <5% of those chips. I have a board with a 16-bit, 16MB PSRAM interface, but then the memory interface is shared between the video generation and the CPU. Going with 4-bit colors or something seems like it would be more in-line with the design philosophy (but it was targeted to be emulated on a modern PC, where memory and color depth are largely free).
Ugh, I just noticed it's even worse - the display is double-buffered, so you actually need 384KB of ram (128KB for system RAM and 2 x 128KB display buffers!).
Implementing a CPU, if you're not trying to hyper-optimize it, is reasonably straightforward. This is a very simple design. It would take a little bit more work to add proper video output and mouse input, but those blocks are easy to find for VGA and PS/2 mice. I think this would run > 100 MHz very easily on anything made in the last decade.
It took me about 20 minutes to make a basic FPGA-friendly implementation of the CPU core in Verilog, but the 256KB of ram is pretty expensive (16-bit color seems excessive for such a simple core!).
That number needs to include car depreciation + any remote 'safety drivers' + the maintenance depot overhead + fuel and insurance + any other costs that scale with the number of cars then.
There is absolutely no way they are remotely close to being revenue positive (unless you're asking strictly if they are charging more than the gas they're burning without counting anything else). They have like 700 very expensive taxis and a burn rate measured in billions of dollars per year.
I think there probably is an actual niche for space-based solar, depending on how retarget-able it is. Imagine being able to beam 24/7 power to remote mining or military installations, or if you could really steer it, beaming it the receiving station with the highest power prices. If it's competing with diesel that needs to be flown in (or trucked through hazardous terrain), it would be much easier for it to be competitive.
The economic side of Waymo still makes absolutely no sense to me. Uber, with no heavy capital investment, has been pretty much unprofitable for over a decade. The car companies are low margin. The taxi companies are low margin. Alphabet has like 30% profit margins. Waymo's business proposition is 1) to own a very capital-intensive fleet of bespoke cars with no resale value, 2) employ an extremely expensive team of engineers to develop autonomous driving for them, 3) use this incredible capital investment to try to undercut an industry that is already barely or not at all profitable and earn back their 10s of billions of dollars in investment (and 10s of billions more of costs in the future), one taxi ride at a time.
And that's completely ignoring their issue with surges, as their robot car supply is inherently fixed - they either don't intend to handle surges, or intend to have low utilization of their fleet.