This is great. I wonder if the price decrease comes from the competition (on the side of anthropic and from local LLMs). If so, I guess we will have to wait for a general GPT-4 competitor to come along before we see price decreases there aswell. Right now it's quiet expensive. We are incorporating it in a new product in the education space and we have to be fairly conservative in rate limiting things so that the cost won't go out of hand.

I also wonder how much of an impact the new Nvidia HGX systems will have on the medium term infra cost on running these services and whether we will see some benefits from that.