True! I just know that model optimization gains are much less guaranteed than say, FLOP density, even though model optimization has so far provided way more gains than hardware advancements.

Part of me is optimistic that when the AI bubble bursts the excess data center capacity is going to be another force driving the cost of inference down.