Also, $1200 for a GPUs that can produce ~7M tokens a day of Qwen 3.8 27b at 80tps is faster and cheaper than any major provider can serve a model of that class as far as I am aware. Pays for itself pretty fast.
Also, $1200 for a GPUs that can produce ~7M tokens a day of Qwen 3.8 27b at 80tps is faster and cheaper than any major provider can serve a model of that class as far as I am aware. Pays for itself pretty fast.
Truly I am not aware of many more complex problems in systems engineering than these, which is why I love working on them, though every new line written is exactly what I would have typed myself when I use LLMs. I mostly use LLMS to help me debug and surgically -delete- dead code and deps to get to results small enough to review in full.
And ~20M tokens a day results in about the max output I can keep up with and carefully review at key checkpoints.
My assumption is the fact my tokens are not unlimited and have ratcheted up one GPU at a time it has caused me to stay much more connected to every line written, but also I am a security engineer working on tech that cannot fail and must be reviewed by a minimum of two humans.
For someone working on video games, I imagine a more lax move fast and break things approach to LLMs might make more sense.
I cannot even comprehend what kind of project could possibly need 3B tokens in a day and still produce results a human could actually hope to review, but do share because I am curious!