13,922 karma · joined September 4, 2011
Another pretty universal thing is that we compare to younger versions of ourselves. That might explain the gap.
When they fired Sam 700 out of 770 OAI employees threatened to move to Microsoft together. So they were giving their work on AGI to MS just like that.
If you go by definition AGI is not general, just "smart ape" shaped.
LLMs are also expensive, not platonic executions. We talk all day long about their costs. Companies developing them are looped with users, investors, competitors and hardware producers. There is a lineage. The self-reference Aaronson can't find in the architecture is in the bill.
I use the /sss writing style - synthetic, short and simple - and it helps a lot.
We are talking about a moving target here ... they get better every few months, so I expect the super-LLMs from 2035 will write amazing code even with sloppy prompting.
I got 400 pp tps on a 10k token input. Your numbers seem suspiciously low, maybe the input was too short to measure properly? And this dense 27B is slow, the MoE A3B models get to 1000 tps.
If you had only batch inference and enough of it to fill the compute to 80% then you get cheaper local models.