Work-sample tests measure the latter.
A worker's experience is a heuristic for programming skill, but a heuristic of a heuristic is like an average of a set of averages: Useless.
I could be plenty wrong though, and I'd love to hear from HR/recruiters about their experiences with cover letters.
Hiring decisions are hard to generalize, because different companies have different needs at different times.
Won't disagree there!
I think that's debatable. The quality of your measurement is then simply a function of how well your tests are in the first place. Do you really think giving a person FizzBuzz is any kind of accurate predictor of future success at a job?
I wouldn't ever use something like FizzBuzz to assess a candidate. It would be more of "here's a mostly finished sample application with a corresponding SQL file, add this feature (e.g. a search bar for a blog) and fix any (intentionally introduced) security bugs you find".
They would be evaluated based on how successfully they complete the main task, and if they have an eye for finding/patching vulnerabilities, that's a bonus that can be used as a secondary selector if a lot of candidates pass. If no one does, it won't be used against them.
That's how I'd approach it, personally. Something specific to the kind of work we're doing, but abstract enough to be approachable without a lot of insider knowledge.
Whether that tells you anything about their ability to do fizzbuzz level programming in a more normal work environment is an open question.
There are a few HN readers who've had experience of "choking" on fizzbuzz level programming tests during interviews - even though it's trivial there's something about the interview situation that causes some people to sometimes freeze.
"Hi, I'd like to ask you to work for a very temporary contract with us so, if it doesn't work out, you have to scramble to find more work and maybe risk homelessness."
See, e.g., http://www.jobsite.co.uk/worklife/probation-periods-19677/
Employment with [company] is "at-will." This means that you may terminate your employment at any time, with no prior notice given to [company]. It also means that the company may terminate your employment at any time, with or without notice or cause. While the company generally adheres to progressive discipline, it is not bound or obligated to do so.
As an at-will employee, you are not guaranteed, in any manner, that you will be employed for any set period of time. No one in the company, except the President, in a written, signed contract, may make any representation or promise to you that you are other than an at-will employee. Any employee, manager or supervisor who makes such a representation or promise to you is not authorized to do so.
For the last few years this has been written into law and employees don't get their full employment rights (such as the right to contest an unfair dismissal) until you've been employed for 12 months. (A de-facto one year probation period.)
I should note that employee rights during this probation period are probably stronger than in many US states.
Them: "Your skills are great, nice job on the coding project. You seem like a really good fit. We want to hire you, but we'd like you to work on a temporary contract with us at first to see how it goes for both of us."
Me: "Hmm, OK, this does seem like a very good fit and I'm cool with the test-drive. My off-site rate is $150/hr, I can have the contract on your desk in two days."
Them: "Er, wha ..., no, you see, we'll take the salary we talked about and just translate that to an hourly rate. It shouldn't be a big deal, this will only be for three months max."
Net of it: after killing it in the interview, I'm offered a short term C2C contract at drastically reduced rates, doing my usual best work, while getting no employee benefits and paying my own SE taxes, retirement contributions, and all business expenses.
I've encountered such propositions only a few times in the past 5 years, and walked each time obviously, but I still find the chutzpah of these companies astonishing.
On the other hand, maybe that was the last part of the interview? They may have wanted to see if I had any self-respect, any self-confidence, could do math, and understand basic business concepts like taxation and fully-loaded employment costs? A "no" to any of these things would have meant I'd be a more ignorant, cheaper and thus much more highly-valued employee as time went on.
Edit: typo
I wasn't arguing in favor of work-sample tests for every company and every candidate, just saying their scope is different (although work-sample tests are more likely to find the untapped potential in the marketplace).