I think that's debatable. The quality of your measurement is then simply a function of how well your tests are in the first place. Do you really think giving a person FizzBuzz is any kind of accurate predictor of future success at a job?
I wouldn't ever use something like FizzBuzz to assess a candidate. It would be more of "here's a mostly finished sample application with a corresponding SQL file, add this feature (e.g. a search bar for a blog) and fix any (intentionally introduced) security bugs you find".
They would be evaluated based on how successfully they complete the main task, and if they have an eye for finding/patching vulnerabilities, that's a bonus that can be used as a secondary selector if a lot of candidates pass. If no one does, it won't be used against them.
That's how I'd approach it, personally. Something specific to the kind of work we're doing, but abstract enough to be approachable without a lot of insider knowledge.
Whether that tells you anything about their ability to do fizzbuzz level programming in a more normal work environment is an open question.
There are a few HN readers who've had experience of "choking" on fizzbuzz level programming tests during interviews - even though it's trivial there's something about the interview situation that causes some people to sometimes freeze.