Although I agree inference will become cheap (probably not zero), it's quite possible that there is a moat they can build around understanding customer context.

With each query OpenAI and Anthropic get more context about their users. When I compare answers between AIs the ChatGPT answer often includes useful things it inferred from earlier conversations. Losing that context could be a meaningful barrier for a customer to switch to a cheaper service.