31,546 karma · joined September 12, 2012
What it's telling you is that a company would rather have 4 people working 5 days a week than 5 people working 4 days a week. The reason for that is, productivity drops a lot when it's spread out over multiple people. The reason behind that is communication overhead - the more context an individual carries in their heads, the less likely their role will exist on an hourly basis in the industry.
So, if anyone wants AI to give us another day off, we need to think about how it can reduce the cost of "context switching" a whole person on and off a task, without simultaneously formalizing our roles so much that it gives us all five. ;-)
Speaking to your point, inference being dramatically less costly than training would not be seen as a delta from the norm. The model of providing inference for anything near the operational costs (like a utility would), would the delta from the norm if it were true.
- The publicly available information about how inference costs compare to training costs is conflicted. EEs involved in datacenters talk about power usage spikes during training runs as if they were a major factor in the designs, but academic papers discussing cost-optimal scaling confidently treat inference-time compute as a major factor.
- On the side of the balance indicating that training is more compute-intensive after amortization than inference is that Chinese providers, constrained primarily by access to compute, have nearly unlimited token availability at a lower price than US providers (inference), but poorer model capabilities (training). That would make sense only if US providers are inflating inference costs by 20-30x due to amortized training costs that overseas providers were not able to take on (there are other factors too).
- If training >> inference, they're in a prisoner's dilemma that far exceeds the ordinary zero-marginals model of competition between firms (due to its huge discrete stepwise nature). On the other hand, if inference>>training, the high-level analysis popularized by certain thought leaders, that it's like a utility, would be true. You'd tend to count this as a vote for inference>>training, but the CEOs saying it at least have a huge incentive to agree because the alternative, the prisoner's dilemma, would stop investment very fast.
- The only voice in the story that I just told you to have anything to do with fact (as opposed to high-level analysis and ivory tower armchair management of a secretive business) were the rumors from facilities engineers. That shows you the state of our understanding...
- If we don't even know the ratio between amortized capital expenses and operational costs, outside investor analysis is impossible. It doesn't matter how finely they divide the accounting buckets for office ferns and indoor ferns if the single biggest part of their business is obscured for trade secret reasons.
"Now he that betrayed him gave them a sign, saying, Whomsoever I shall kiss, that same is he: hold him fast. And forthwith he came to Jesus, and said, Hail, master; and kissed him. And Jesus said unto him, Friend, wherefore art thou come? Then came they, and laid hands on Jesus and took him. And, behold, one of them which were with Jesus stretched out his hand, and drew his sword, and struck a servant of the high priest's, and smote off his ear. Then said Jesus unto him, Put up again thy sword into his place: for all they that take the sword shall perish with the sword. Thinkest thou that I cannot now pray to my Father, and he shall presently give me more than twelve legions of angels? But how then shall the scriptures be fulfilled, that thus it must be? In that same hour said Jesus to the multitudes, Are ye come out as against a thief with swords and staves for to take me? I sat daily with you teaching in the temple, and ye laid no hold on me."
>So it's in theory possible that someone could make millions tax free from selling a business, but that's a rare case and one the tax code explicitly allows for.
These guys have whole offices working to figure out how their day to day expenses can be converted into stuff like "selling a business." ;-)
P.S. a wealth tax is a property tax. They have existed in the US since before the income tax (which was originally considered unconstitutional by its opponents).
The idea is simply that the basic idea behind LLMs, that you're distilling the entropy out of the entire available world of text, is antithetical to creativity.
Further developing on the theme of self-play, humans have the ability to sense what we want (intellectually) and reach for it communally over thousands of years. It's an innate quality, and if AI starts participating (contrast to giving people psychosis) we will all be able to tell.
In this model, the exponential growth that everybody is freaking out about is only the realization of the modular software dream ("we'll only have to write an ORM once for all of human history!") and the sheer amount of knowledge in libraries.
It's at least falsifiable.
In reality, if you have 100 5% chances of a cure for a previously incurable illness, you can celebrate each chance a lot.