It a small company not specialized in education and examinations can set a test and deduce if I am good enough to code, then this can be standardized. And there could be many certs: 1. can code, 2. can code c++, 3. understands multithreading … and so on.
Now i for one dislike leet code and coding challenge interviews. I think it's silly, plain and simple. What i would look for instead if i was an employer would be curiosity. Nothing is more important than insatiable curiosity. A curious employee will learn everything about your whole stack in the first week... just out of curiosity. And if they do hit a leet code like problem during day to day work you best believe their curiosity won't let them rest until they've solved it, be if with prior knowledge and experience or without.
He was unable to execute a .py file.
A PhD doesn't mean that a person knows everything. It means they made progress in understanding in a narrow area. Might not have included Python.
It's almost certain some of then (maybe most of them) would have fared just fine if they were left with the problem alone, not in a high pressure setting like a job interview.
I don't know how you test for the ability to do real world stuff - I always think of doctors, they have a system of proctoring they've developed over the years, where you're judged by your peers and rated accordingly and even then its not 100%. I think this is the only way to do it in real life, but I don't know if programming will ever get to that point, it probably will be necessary sometime - when everything is driven by computers.