Hacker News Reader

Comment by raghavsb | Hacker News Reader

The talk title is quite provocative but the material discussed not so much. It is true that core data science is iterative, needs to reproducible and more recently explainable. Can you do everything in GUI? It really depends on where you see it from. Early data scientists were programmers before so they loved to code and built tooling around it. While code is still dominant we also see the rise of UI-centric tools - these allow you to build ML pipelines by snapping blocks together. I feel are chasing a "different" type of data scientist. The term data scientist itself has become quite broad.

Code, CLI gives data scientists infinite flexibility but setup, management etc. is a challenge. GUI provides very less flexibility but you will have output fast - works for simpler problems IMHO.

GUI or not data science has to move to cloud-based tools. Whether you write code in browser or CLI on local machine is matter of choice.

View on HN

> GUI or not data science has to move to cloud-based tools.

This is a pretty odd statement. Care to explain why?

Smaller datasets will work on laptop/desktop. For DL work with large datasets you need to build a GPU workstation. Moreover there setting up environment and dependencies on different hardware setups is not straight forward.

Cloud provides the flexibility of choosing the hardware, many open source projects allow you to manage your dependencies and setup better. From 0 to something, cloud is better than custom.

For quite many companies their whole "big data" dataset is small enough to be processed by a single beefy machine. In that case it's far more cost effective to simply plug in $1000 worth of extra ram in a workstation rather than spend some extra engineer-hours to do it remotely.

Couldn't agree more on the data size. In most cases beefy machine work. Would on-demand (cloud) make it simple?

Also beefy machine works for training jobs. But we need to deploy the models too.

With the many container solutions available today it's incredibly easy to move from dev to prod. You don't need to pay for a prod environment to do your development just to avoid ever having to migrate.

Yes, it is incredibly easy, except when you upgrade to tensorflow 1.6 and it fails with [a cuda error][1] and after couple of sleepless nights you realize nvidia has deleted the docker image of cuda version 70xx from dockerhub and you need to find the right commit that works from their git repo and build everything yourself.

[1]: https://github.com/tensorflow/tensorflow/issues/17566

So if you struggle to "setting up environment and dependencies on different hardware setups" then maybe Data science and technical programming is not for you.

I wish this view was held more widely. Unfortunately the vast majority of "data science" jobs out there will not ask for any such technical ability.

Reply on news.ycombinator.com