We are waiting to launch a new iOS app that has text generation using vertex AI for GA. So we will go live next live.
We started with GPT API but switched to Vertex AI due to speed. We will still use GPT API as backup still though.
We started with GPT API but switched to Vertex AI due to speed. We will still use GPT API as backup still though.
1. text-bison limited to 1024 output tokens.
2. Output format we ask for JSON. But it is not valid json many times (, after last element, missing } after element etc). We have to write our own parsing code in the end to work around these JSON format issues.