In my experience it is comparable with chatGPT 3.5 but is more expensive (I still use it cause I hate guardrails).
Does mistral not have any guardrails?
Mistral API has a flag in the json payload to remove guardrails.
Nope! It’s actually for adults to make their own choices and not some SV public relations firm nerfing it.
You can buy a good GPU and quickly come below cost for GPT APIs depending on your workload. I’m doing millions of tasks, so eating $8k on a custom build for ML up front is a long term cost savings. Not least of all that every other month there’s an even better model.
You'd need high usage density to justify investing on a GPU just to run a 70b model locally.
I don't see myself doing millions of tasks on 70b+ models. Work pays for more gpt4 that I can chew; I can run up to 30b models on my mac with a decent speed, and I can use mistral APIs when I travel without powering my own home server. I can see how that buying nvidia GPUs would make sense for a heavy user... Also, I had a gaming addiction for years and try to stay away from anything that can be used for gaming.
I’m bootstrapping a startup, so it ends up being cost effective. I also think the long-term outcome is open models will win, so I’m betting on that future while also maintaining OpenAI compatibility in the present.