I agree with you in principle, but one thing to keep in mind about the medical community is that there is a legitimate need for broadly diverse medical professionals in a hospital. The reason is for community outreach -- someone of the same background (say, socio-racial-economic) can communicate with the patients easier and have a better outcome for the patients' health.
I am sympathetic to the desire for the best qualified people and affirmative action truly does scare me, but medicine does need a bit of special consideration here, in my opinion.