Very cool, thank you for putting it together and sharing the data
Would you mind adding Jeffy to your benchmark?
https://github.com/nicobrenner/jeffy
https://jeffyclassify.com/blog/when-llms-fail-at-classificat...
Would you mind adding Jeffy to your benchmark?
https://github.com/nicobrenner/jeffy
https://jeffyclassify.com/blog/when-llms-fail-at-classificat...
Thank you for giving it a go. Will have to put some time into how to got about it
You are correct though, and it’s an interesting property of training classifiers, that you are essentially encoding the preference of the data labeler(s)
Looks very cool though. I’d definitely like to somehow “generalize” Jeffy to be more of a universal classifier. Although adhoc models pretty much always beat general ones at classification