I think hardware scarcity plays a part in inference margins. They would get squeezed super quickly if anyone who wanted could just build an inference cluster, but that's not currently the case.
We're also starting to see efficiency gains, so we might meet halfway.