I like your idea from a company's perspective. I've seen many teams struggling to get good developers - Part of the problem is the existing team (it sucks technically) - they can't ask the right questions and end up hiring the wrong kinda folks.

You should probably explain a bit more in detail (on a blogpost maybe) on the matrix that you will use for evaluation related to techs.

I agree with few of the comments below too.

1) I'll not be interested in talking to you if it seems to me that you are a recruiter. But then - its me and people like me who are humbly confident about their skills - that's mostly not the case with everybody. May be you could partner with an employee -mostly silent- during interviews.

2) You could be doing a great deal of misdeed towards developers. I judge my prospective employer/team based on the questions they asked, how they asked, etc, etc. I've been trapped into a situation where I thought that the team is "very good" based on the interviewer - which turned out to be a disaster - bcoz the interviewer was brought in from some other team (read: excellent team) - but that's not what I was hired for.