Ask them to explain what a SVM is. Ask them to explain how training a linear perceptron works. This kind of stuff.
I can guarantee some will pick and reject people because they did things in a way they didn't like
Two years later, I interviewed at a large SV company, and they asked what projects I'd been working on. I gave a description of this project (2 years prior) and my explanation was phenomenal; in order to understand SVMs two years prior (without the 'necessary' mathematical background) I needed to develop all of the intuition (up to 'is kinda a high pass filter' etc) (which you might not ordinarily do at a time-pressured university course).
The interviewer was correspondingly impressed, and the SV company gave me an internship almost directly off the back of this interview.
The kicker is that at this point, I had a rudimentary knowledge of linear algebra, and absolutely no knowledge of any other machine learning; I had no business interning in their data science team.
My point being that even as a first pass, the bookwork questions don't work fantastically. FizzBuzz is no better, but a data science alternative would have weeded me out pretty quickly.
(If you didn't, maybe the interview worked...)
Apologies in advance for the bluntness, and don't feel like you need to answer.
One way it happens is that you get a PhD in astrophysics with years of data analysis experience in for a data science job. Have a software engineer interview her and he might find that she doesn't know a number of basic computer sciences concepts [traversing a linked list, tail recursion, implement breadth-first-search]. His knowledge background says these basic ideas are fundamental, there are therefore serious questions about the technical ability of the interviewee.
Or in other words...I'd be skeptical if a candidate hadn't learned programming on their own even if it wasn't required because it's pretty much impossible to get any practical experience otherwise.