Hacker News Reader

Comment by alexchantavy | Hacker News Reader

Is there a commonly accepted standard for qualifications of a data scientist, are they expected to be software engineers with PhDs in something statistical/data-oriented? I've heard mixed things and am interested in getting clarity on this - sometimes they have CS backgrounds, sometimes they have non-CS backgrounds, some are competent programmers, others can't write a line of code, etc.

View on HN

Nope is the short answer. But in general you're expected to have a grasp of the fundamentals of statistics, some software engineering, Python and/or R, knowledge of the various algorithmic approaches, and a host of traits which really make a data scientist: curiosity, persistence, determination, flexibility, adaptability, detail oriented, conscientious, really like a challenge, resourceful, good interpersonal skills, ability to convey the complex in simple terms... In many ways I think it's almost more about the traits than the training.

But yeah, where I've worked that's generally what we look for in candidates.

What is astonishing to me is how there seems to be 1) a dearth of candidates, period, and 2) candidates we can dig up miss scheduled calls, show up late for interviews, interview very poorly, turn in poor quality take home exercises (an exercise which essentially just covers the basics), have really crappy resumes (typos, horrible layout, inconsistencies with LinkedIn profile, etc...)--and these are folks with experience as statisticians or data scientists. Amazing.

We don't ask anything deep or complex either, yet we've had a really hard time finding people.

I don't know much about hiring (so my opinion isn't worth much), but I assume you're using "data scientist" as the job title? If that's the case, it might be the reason for some of the difficulty you're experiencing. The way I see it, "Data Scientist" as a job title has only been around for ~8 years, and there is currently way too much hype surrounding the phrase. I've seen a lot of posters on HN and elsewhere acknowledge that they are trying to land this type of job just for the title, in order to get it on their resume.

I would be interested to find out if you are using the phrase, and what would happen to your search for candidates if you changed the title to something less "sexy"[1], like "data analyst"?

forgot the "sexy" ref: https://hbr.org/2012/10/data-scientist-the-sexiest-job-of-th...

The chief data scientist at my last job said a Data Scientist knows more programming than your average statistician and more statistics than your average programmer. There's this venn diagram he used to show with the intersection of skills for the different disciplines involved - some math, some engineering, some communications.

I think there's also an intersection with devops skills, maybe less important, but your hardcore statisticians usually put zero thought into operational considerations. Really the last bastion of "works on my machine" thinkers in the computing world. I just finished the Coursera "Reproducible Research" course and I was really struck how many of those principles parallel good software engineering practices - use source control, document through code, separate your environment from your code, automate as much as you possibly can, etc. I've been a software engineer for 20+ years but I want to get into data science partly because I've always been a data head, just without the theoretical background to do really interesting work, but also because I think I can bring some of the software engineering skills to bear.

Also, with grading peer's work on Coursera, I really realize that a lot of these candidates need help with their English and presentation skills. Many of the students put no work at all into the presentation, I imagine that's going to serve them poorly in the working world.

The way I've heard it from others in this forum, is that a DS is a combination of three jobs. They are analysts, in that they can work with data and squeeze insights out of it and they know enough about the business to know that the numbers mean and what differences matter. They are software developers, in that they can build actual software solutions to access and manipulate data, rather than relying purely on shake-n-bake existing tools. This helps them deal with very large data sets that are beyond conventional analyst tools such as spreadsheets. And finally they are experts in stats/ai/math who can build and evaluate sophisticated mathematical models.

It seems to me that's an awful lot to fit between one pair of ears.

Thanks. From what you've seen, what kind of background experience (education, line of work, etc) is necessary for a candidate to even get an interview (i.e. things that you see on a resume)?

Reply on news.ycombinator.com