Ask HN: Burned Out Data Scientist – Switching to Back End SWE? | Hacker News Reader

Ask HN: Burned Out Data Scientist – Switching to Back End SWE?

9 pointswltrmcq13 comments

I am a burned out data scientist in a data "head" role at a startup. At this point I have management experience and am at mid-career - I have been doing the data thing for a decade. I can't do it anymore, I can't stand cleaning or working with dirty data on a daily basis. I am sick of managing expectations about DS work with managers who just want models and results when they're not tenable.

I have started to learn how to write back end code e.g. web services, APIs, working with Apache Kafka, and have found that working on these types of projects brings me joy. I am worried that companies will look at me as a very odd choice for hiring because of this career switch. Has anybody else gone from DS to more vanilla SWE? If so, what was your experience doing so? Any advice? Thanks.

View on HN

> I have started to learn how to write back end code e.g. web services, APIs, working with Apache Kafka, and have found that working on these types of projects brings me joy.

Similar to data science, these things are fun when you play with them on your own. In a company setting however, it's often a shitshow not so different from your bad data situation. I'd be cautious about switching.

100% this. You will almost without a doubt experience the same sort of frustrations with software engineering, whether its in a marketing agency/dev shop or a product based shop.

I also agree with the warning from burntoutfire to be cautious above.

I am similar to the OP in background and interest in job change but have decided at this stage of my career that I am FAR more interested in joining a small team that I am excited to work with than being super-focused on a specific job role or tech stack.

I also suspect the OP would have an easier chance of transitioning to managing a software team than a different SWE role. I think hiring orgs would see that a more "moving forward" step than a sideways move into some other SWE gig.

Agree. You might be better off sticking with DS and finding a different company with better management. Even as a SWE your satisfaction of that job will probably depend on the company and manager that you work for.

I did it two years ago after a couple of years of being a data scientist. I got tired of data cleaning, experimenting, tuning parameters, and managing unrealistic expectations about DS work. Data Science tasks seemed open-ended, I wanted something more deterministic, tasks that I could call finished at some point.

So I went from Data Scientist to Data Engineer which, in my current company, is a SWE working on data (as opposed to full stack or backend engineers). In my new role I worked closely with data scientists to ship their code to production, as well as data pipelines and some backend and infrastructure tasks. I'm enjoying it so far. It gets repetitive sometimes, but more bearable than DS work.

If you have enough SWE skills, as well as DS knownlegde, your profile could be attractive to some companies. You can also target ML Engineering or Data Engineering jobs. I think you don't have to worry much about switching. It's just a matter of finding the right company.

Good luck!

Are there any courses/certifications that you did in order to complete this transition?

Not really, I was self-taught most of it by reading docs, code on Github, etc.

Have you considered that maybe it's the head part of the role that's burning you out? I don't think you'd have much trouble switching, but I'd try out being an IC data scientist at somewhere with good work life balance first.

I'm in a similar situation but with less experience than you. After a couple of years of being a data scientist I'm realising that I enjoy building stuff a lot more than the other aspects of the job. So for my next job I'm looking for an ML engineer position - that way I can do more of the kind of work I like but it also means that I can leverage my existing knowledge and experience and I don't have to start from scratch.

I am also on the same boat, albeit at a lower designation. I totally agree with the bit about managers wanting results without having any clue on the feasibility of projects.

Are there any IT careers which would be an easier transition apart from ML Engineer roles? Any thing based on Cybersecurity perhaps?

It’s a good move. SWE offers much better career prospects for very little effort. With a DS background you will have an edge also, provided you have the aptitude to do SWE. (A lot of “DS” don’t.)

Do you have any advice for people going in the other direction, wltrmcq?

be sure that it's a data science and not startups that you're burnt out on

Reply on news.ycombinator.com