8,170 karma · joined December 20, 2010
https://art.cx
https://revise.io
I had my first kid accidentally in college and dropped out to focus on that. Very grateful for it.
Point taken on using an agent. I went that route because part of the goal for this benchmark is to inform which models I push in my agentic word processor, which uses tools for focused proofreading/editing. It's much faster and generally cheaper to use tools for surgical changes on large documents, rather than having the model spit out the entire document with all issues corrected. So yes, I am trying to measure agentic abilities here.
A simple one-pass full-rewrite test would also make an interesting benchmark, though.
I've been working on this to inform my own decisions about which models to use in my agentic word processor, but I think it's also just useful data.
I just ran GPT 5.5 and it broke Gemini's previous high score of 92.5%!
The code and run artifacts are available on Github: https://github.com/reviseio/errata-bench