Why don't they just let project maintainers say, "This project contains x, y, and z" or something? That'd at least let them get a leg up on doing the categorization right and I don't think many people would mind having that capability.
It's probably also a bug to even have the notion of "a language" for a repo given the burgeoning polyglot programming trend. So many repos these days contain multiple languages, especially when you consider javascript, that I question if it even makes sense to say 'This project is in language X' at all.
Like you say, the best option really would be to let the repo owners / maintainers just specify this stuff. They are, after all, the ones who know.
Note: I'm not saying they shouldn't have the auto-detection, because it definitely helps if the maintainer doesn't do it, but for those that want to help classify things - let them!
"Sorry, I couldn't determine if you had C code in your repo or is that Limbo code?"