Also, the experience of the less privileged is that some people use claimed-impartial tests and barriers to exclude them in particular, and so when a test comes along it is demotivating, as it might just be an attempt to exclude them. Look at the voting reforms in various places for examples of this.
Using tests to try for pure meritocracy only works when everyone has had the same kind of background and nobody has bias. Look at the examples of CVs/books sent around with just the name/gender changed and the huge difference in response rate to see the bias. Essentially, if you're recruiting (or filtering any set of people) and aren't actively controlling for bias (anonymization is key), you will be biased.