binalpatel

The market value (i.e. the big bucks) I think will shift into Data Engineering and the role that's abstractly called "Machine Learning Engineer".

Reliably getting any data science analysis or model running in a real world setting is a demand that's naturally going to follow from the Data Science glut.

I work in healthcare and one thing that's always top of mind for me is that that the data we work with is only (generally) indicative of the physicians'/billers' best guess.

So - say we get a bunch of diagnosis codes from a hospital. Codes are generally added in via medical billers after a patient is discharged, based on the physicians input and other data on the patient record. So at this point the data in which you generally work with has gone through two different humans, who used their (best attempt) subjective viewpoint for this patient.

This generally works fine for a lot of common conditions, evident conditions, so things like heart attacks, fractures, and so on. But for things that are complex, like sepsis leading to more evident conditions, the data may not necessarily capture that sepsis even occurred.

Not to say this is a unique problem to healthcare, but something that's not talked about often. A lot of the data we train and model on is based on a human's best guess, which may in some ways be limiting given really complex, dynamic processes.

At a former employer someone created a framework to try and create a generic process to process and load data (i.e. ETL). It was written with the best of intentions, but was terrible.

This framework was given to an offshore team, and they were told to use it to farm out hundreds of requests. The framework was inflexible enough that they each started adding snippets here and there to make it work for those projects, with no code review.

When I joined there were well over a hundred different projects, all using this framework at the core, most having little bespoke tweaks to make the code work. Every failure was essentially a new failure in a new way.

It was a useful experience - it's one of those experiences that teaches me via a negative example. This was the worst example of how "roll your own" resulted in incredible technical debt I've seen.

I find conda far and above the best tool to manage python packages and dependencies. Being able to concisely contain all Python and binary dependencies together is invaluable.

I recently wrote about it as blog post, using conda within containers has solved almost every pain point we had with python packaging and how to get things into production reliably.

I'm a big fan - I've been using their old version of their library for NLP on medical text, but the course and library certainly removed a lot of the "mystique" around deep learning for me.

"Musk and Tesla must each pay $20 million in fines, which will be distributed to traders who were harmed by the tweets."

At face-value this sounds like "distribute the proceeds back to the hedge funds that lost money".

Scikit Learn comes to mind - not just because I can dig into the source code and immediately know what's happening, but also for the stellar documentation that goes above and beyond telling your what the functions do.

For example their Cross Validation documentation is amazing:

http://scikit-learn.org/stable/modules/cross_validation.html...

Related article on the same website: https://thegradient.pub/nlp-imagenet/

NLP (or specifically NLP using deep learning) seems to be having a breakout moment in the last year or so where there have been large advancements back to back.

Generalization is hard - you're often tuning millions of parameters at once, and often the most "sane" thing for the loss function to do is rote memorization. It'll be interesting to see what comes about from this discussion.

There's this great article I came across a few weeks ago:

https://www.theatlantic.com/science/archive/2018/03/human-ex...

About how we may expect life to be more prevalent than it is because our own existence is so miraculous. Though the other side is - if the universe is so large, with so many planets, then there may be a chance that there's life out there that's just as incredibly lucky as us.

I love thinking about the sheer scale of it all. This is a billion stars of the ~200-400 billion stars in our galaxy. The Andromeda Galaxy (the closest to us) has ~1 trillion stars.

There's an estimated 100 billion planets in the Milky Way alone, and who knows how many in the Andromeda galaxy.

There could be another satellite somewhere in this galaxy charting space just like this and we'd just be a bright dot to them among billions of other bright dots.

This seems like Facebook's trying to break more into the realm that the big credit agencies have occupied re: data products.

All the big credit agencies (Experian, Equifax, Transunion, etc) have products that cluster people into socioeconomic groups, with ridiculous names.

https://www.experian.com/assets/marketing-services/brochures...

Examples from that brochure:

"American Royalty" "Small Town Shallow Pockets" "Full Pockets, Empty Nests"

Thought I'd repost this - given it took effect on January 1st. Employers can also no longer ask for, or try to obtain, salary history as well.

Wow - I never realized the scale of the Mt. Gox one. Wiki says they lost 855,000 BitCoin, which at current prices are around $8.5 billion USD.

Reminds me of a talk I saw by Stephen Boyd from Stanford a few years ago: https://www.youtube.com/watch?v=wqy-og_7SLs

(Slides only here: https://www.slideshare.net/0xdata/h2o-world-consensus-optimi...)

At that time I was working at a healthcare startup, and the ramifications of consensus algorithms blew my mind, especially given the constraints of HIPAA. This could be massive within the medical space, being able to train an algorithm with data from everyone, while still preserving privacy.

It still blows my mind that we can detect planets orbiting an entirely different stars. And more so - that these tiny pixels that we can just barely detect, are full-fledged planets that may have atmospheres, water cycles, and who knows what else.

You work in an industry like healthcare where it's IPSEC or bust for any tunneling.

I agree with you - coming back and rereading the article, I didn't have nearly the same reaction. Originally I focused exclusively on that single segment, which my fault.

The glorification of stress in start-ups is an interesting thing. I've always wonder if it's because it's a mental thing. If an industry glorified their employees physically harming themselves, it'd be horrific.

It took me a while to learn, but working hard doesn't equate to being stressed, and being stressed is more than likely to result in mediocre results.

I'd argue this challenge was one of Netflix's better investments.

Sure - most of the winning algos never actually got used - but here we are, still talking about this almost 10 years after it's inception.

I've fallen into the busy trap pretty often lately.

Busy to me - paradoxically - is correlated with not getting shit done. It's one thing to have a lot to do, and get 90% of the tasks done, and another to have a lot to do, and get 10% of the tasks done, with the other 90% rolling over onto the next day.

It's the roll-over that's really the stressor for me, when I feel most "busy", and that creates a self-fulfilling prophecy for the next day, when you have that much more to do.

I'm in the midst of a light burnout right now - and I realize it's partly my expectations, and partly the expectations of my company.

As part of a start-up you're expected to be insanely productive at all times - so much so - that "normal" productivity feels like you're spinning your wheels and not getting anything done.

The other side is the company encouraging being a "hero". Being the one who can do anything, who gets X, Y, Z done on time, even if it takes all night. The praise and recognition fades quickly - as you realize you've set the expectation such that you can function at 200% at all times. (And you start feeling stressed as you fall to 150%, to 120%, to 100%.)

Some of the comments in this thread are spot on. For the level where I'm at (not full-bore burnout), I'm trying to prioritize my life outside of work again (aka have a life outside work). Sleep well, exercise, pick up hobbies again, go for hikes again, and so on.

Burning out to me - is letting work take up a disproportionate amount of your time and mental energy - so much so that the microcosm of work starts feeling like the most important thing.

Getting out of it (for me) is all about regaining that perspective.

I completely agree on that - it's exactly like math, the only way to truly learn it is to practice, over and over again, until you've mastered the concept.

This is something I'm coming to terms with in my career right now (~3 years in) - do I learn something to a mastery level, even if it's not the most marketable thing, or do I focus on breadth first and shoot for depth later.

This is employer-centric in my opinion - sure you're not paid to write code, and sure you can reuse existing code base in new and exciting ways.

But - there's also value in learning new things, in new skills, in new toolsets. Maybe it's not best for the employer, but for you the employee it can help quite a bit.

There's a fine balance - ideally your employer encourages you to learn things on the job, and to try new things that may not necessarily make it to production, but improve your skillset overall.

On a - I have no idea.

On b-

Not sure if it's worth $25 billion, but I have noticed that it is the "Facebook" for younger generations, and that when they use it, they use it all the time, not just a few times a day like other social media. It's a way to share their lives - in real time - with their friends and followers, and have their friends do the same.

(Look at me sounding like an old man!)

I'm quite frankly amazed ICQ is still alive, I thought it was one of those things that had just died out some time ago.

I do quite enjoy that survivorship bias!

I wax poetic about this pretty often - but it's always been amazing to me that me, and you, and everyone in the world has an unbroken chain of ancestry all the way back to the first spark of life.

We're all stupendously lucky in that regard.

404 Page for me.

Though I must say - that might be the coolest 404 page I've ever seen.

Edit: The 404 page in question - in case the article actually comes back up :). http://imgur.com/a/cbhS2

Edit2: The actual link to go to: http://www.theverge.com/2016/11/14/13621108/hayao-miyazaki-l...

I think this is an important point - geopolitical issues are part of the equation. When Russia was invading (or liberating, based on your viewpoint) Kiev, energy reliance on Russia did play a key part in some nations willingness/unwillingness to take part in sanctions/retaliation.

Given that - if 90% of a shared resource is controlled by one or two large entities (made up numbers), that puts you in a weaker position as one of the smaller entities in the 10%.

Maybe I'm generalizing too much - but couldn't we say that electricity generation is starting to resemble the internet more and more?

Servers (generators) that provide content, users (consumers) that want to consume it. As more and more people use things like rooftop solar - more servers are brought online, and the infrastructure needs to be updated accordingly to direct the "traffic" well.