this was announced last week. The major changes are memory capacity and bandwidth. That means smaller batch sizes (i.e. inference) will be faster (1.4-1.7x) and larger batch sizes (i.e. training) will be possible. Current large batch sizes, however, won't be faster.
That assumes there will be a ChatGPT left at this point.
Microsoft has the GPT4 weights, ChatGPT will be available in some form regardless of what happens to OpenAI.
And they just introduced their own AI chips at Ignite.
They might have the weights, but for some reason the responses from the microsoft chat is much worse quality for me than gpt-4. Even the local 180B falcon I run locally seems to be better in most cases.
If openai goes under, I personally hope they release all the weights, source code and training data to public domain.
What's "the Microsoft chat"? I've got multiple GPT4 models deployed in Azure and I can't see a difference to vanilla OpenAI's responses.
What hardware is OpenAI using now for running ChatGTP? And how many users can they serve with it simultaneously per instance?
It would be interesting to know the full architecture of ChatGPT infrastructure. How they are able to scale such a compute and memory hungry model to 100 million users.