Those are still numerical scores and, as far as I know, essentially based on “does it solve the task correctly or not”. By “qualitative”, I meant for example some analysis of how it succeeds (e.g. quality of prose or code) or its failure modes (lack of persistence, etc.).

This version added two private sources which cannot be inspected.