This would eliminate a lot of cheating, and a lot of advantages for those in a good position (better access to new drugs and nutrition, better recovery programs, better training programs — aren't they all unfair at some level?)...
The ultimate goal is to get us to experience the top level combination of talent and effort, both in science/work and otherwise. Getting there is never going to be completely fair (hey, you scored better on it even though you prepared for 3 days and I took 30: tough luck for me, I guess, but the fact you are more talented for that exam is not something I can do anything about).
I've also seen non-cheating people who are excellent at exam taking (great scores) without ever taking anything from the actual material (zero learnings). I've never felt threatened by them either, though maybe I would have if I wanted to pursue an academic career.