Sure. This thing I propose might already exist in a nascent form.
I haven't checked in a few months and the way things are moving now...
Interesting!! I'll talk with my brother about this - sounds very cool!
Hey Daniel, I would love to help out on this. I'm learning about LLMs and this benchmarking project sounds like a fun way to further my knowledge and skills. I sent you a message on LinkedIn.