Even putting aside the whole privacy and tuning debates - unless Sam is willing to sell us inference at something near the pre-constrained cost of directly owning the DRAM all he is advocating for is "don't be greedy guys, that's what I'm for. I can be greedy for you!". It's not actually efficient for people to pay huge markups on compute and memory vs letting that same memory site idle some of the time. I mean I guess it's efficient for OpenAIs bank account, but not for anyone else.