This is a temporary problem. A 10x-100x speedup will allow us to simply wait for the response, instead of finding something else to do in the meantime.
I wonder what the experience is of people here with the ultrafast mode (8x) of Codex.
I wonder what the experience is of people here with the ultrafast mode (8x) of Codex.
I've tried to find a combination of a good-enough model, a VERY fast provider, and cheap enough serving to make this all doable. DeepSeek itself is quite fast, but I crave even more speed (and ideally something hosted outside of China).
Fireworks and Baseten seem to be the leading providers that are available to a solo dev like myself, and support any reasonably decent modern models (DeekSeek 4.x, GLM 5.x, etc). Fireworks has been a real dud, but Baseten in FAST mode has been absolutely amazing. Not cheap - but I find it substantially cheaper than Terra / Sol (low), etc. I haven't found Haiku to be smart enough to be useful.
DeepSeek (the provider) and beyond seem to hit the sweet spot to keep me mostly in-context, so I don't need 4 parallel work items to feel productive. So roughly 200tps or greater. Vanilla Baseten is ~400 tps, and FAST is 700-800 avg.
Price wise, Deepseek hosting is extremely cheap, Baseten is a step up - maybe 5x or more, and FAST mode is well beyond that. But I still find myself dual wielding Deepseek and Baseten FAST when I need the speed.
The other benefit: for my own work in very limited hours, I'm more than happy to pay for answers to come back faster.