214 karma · joined March 25, 2012
https://sg.linkedin.com/in/mailshanx
Not sure if this is meant to be satire / sarcasm?
By "classically" trained statistician, i meant someone who has taken a bunch of grad level courses in statistics (as opposed to CS flavoured ML).
A lot of Fisher's and Gosset's work was done in an era of "small" data.
>>if you can figure out how a "classically" trained statistician can create an acurate model with:
>>1) almost no data
>>2) inaccurate data
>>3) seasons in which the next one often bears no resemblance to the previous one
Stats / ML can't help much with 1) and 2). But wrt 3), dealing with issues of non-stationarity is very much a subject of statistical literature.
Here is where a classically trained statistician can really shine. Classical statistics is really all about extracting insight from data when it is hard to come by...
Top 2% world rank on Kaggle.com. History of building a number of data driven products and systems, some of which have resulted in a 13x improvement in key business metrics. Speaking engagements at conferences such as Europython, published research in machine learning and advanced degree from the top ranked university in Asia.
Location: Singapore.
Willing to relocate: Yes.
Stack: Python Data Stack(Numpy, Pandas, Scipy, Scikit-Learn).
Java, Hadoop / Hive / Spark.
Contact: https://sg.linkedin.com/in/mailshanx / mailshanx at yahoo dot co dot in
Get in touch with me, i'll be happy to talk to you :)
Also think of your financial goals - 1M brings you pretty close to FU money.
May your mother rest in peace. tears in my eyes
Top 2% rank on Kaggle.com. Built the machine learning engine for the world's fastest underwater modem, leading to a 1300% performance improvement. I speak at conferences such as Europython and have an advanced degree.
Location: Singapore. Willing to relocate.
Stack: Python Data Stack(Numpy, Pandas, Scipy, Scikit-Learn), Matlab, Java.
Contact: http://shanx.us / mailshanx at yahoo dot co dot in
Get in touch with me, i'll be happy to talk to you :)
I've been following your writing for quite a while now, and i love it :)
Heartiest congratulations again :D
1. Probability and stats: For probability, work through John Tsitsiklis's MIT OCW course 6.041. There is also an archived version of the course available on EDX. Norman Matloff's book (http://heather.cs.ucdavis.edu/~matloff/132/PLN/ProbStatBook....) approaches stats from a CS practitioner's point of view (lots of sample simulation code in R), and has intuitive explanations of many different statistical concepts. For Hypothesis testing, checkout the courses Mathematical Biostatistics - 1 and 2 on Coursera.
2. Linear Algebra: Gilbert Strang's OCW lectures / course notes is hands down the best resource. But depending on your mathematical background, the lectures can be a bit long-winded.
3. Programming proficiency: For algorithms, Robert Sedgewick's algorithms course on Coursera has excellent videos. His textbook is very well written too. I really like his style of writing actual working code for each algorithm he describes (in contrast to say, CLRS): i find his code samples rather elegant and worthy of study. For learning object orientation and design patterns, the best way is to write large programs and read the book "Head First Design Patterns" as you do so.
The aim is to develop an ability to think with a statistical / algorithmic lens. Many real world phenomena are stochastic in nature. Modelling them by simulation and measuring and drawing confidence intervals around your measurements must become second nature. When confronted with an image / text processing problem, representing you data as vectors and manipulating them must be natural. While writing large programs, being aware of design considerations such as preferring composition must come naturally to you. Of course, you should be able to always recognize when a computation is O(n^2) vs O(n log(n)) vs O(2^n).
Once you are done with that, all you need to do is work through Andrew Ng's Machine Learning course and compete in a few Kaggle competitions / build some ML related side projects. Congratulations, you are now a data scientist good enough to work at most places in the tech industry!;)
It is much, much more important to master the foundations, which consist of: 1) Probability and statistics 2) Linear algebra, and 3) Programming proficiency
To be more precise here is what you would need to know in each area:
1) Probability and stats: Properties of the most common distributions (Normal, Poisson, Exponential), conditional probability, properties of expectation, Bayes rule and its applications, hypothesis testing, confidence intervals, bootstrap / jackknife, know how to compute the power of a test, be able to run Monte Carlo simulations to model real world phenomena.
2. Linear Algebra: Know enough to understand and compute Eigenvalues / Eigenvectors, SVD and various matrix factorizations.
3. Programming proficiency: Algorithms and design patterns. Its enough to know the typical undergraduate algorithms course content: trees, heaps, hash tables, various sorting algorithms, graph algorithms: shortest paths, spanning trees, max flows.
If you know this, you can easily pickup everything else on a need-to-know basis. Depending on the domain you are working in, that might entail either natural language processing, computer vision, machine learning or big-data tools like Hadoop / Spark etc. All of that is very easy to learn once you have the foundations.
Me: "That will depend on the other things in the compensation package, such as the paid time off."
HR: "Everyone gets 20 days. We have gym, swimming pool, and medical insurance. We don't offer stock and don't give bonuses: it is a monthly cash based salary structure. Now tell me, what's your minimum?"
I ended up spitting out a number. How should i have responded ideally?
I'd venture to suggest that most people know this, but lack the confidence / daring to do so...
Top 2% rank on Kaggle.com. Built the machine learning engine for the world's fastest underwater modem, leading to a 1300% performance improvement. I speak at conferences such as Europython and have an advanced degree.
Location: Singapore. Willing to relocate.
Stack: Python Data Stack(Numpy, Pandas, Scipy, Scikit-Learn), Matlab, Java.
Contact: http://shanx.us / mailshanx at yahoo dot co dot in
Get in touch with me, i'll be happy to talk to you :)
Top 2% rank on Kaggle.com. Built the machine learning engine for the world's fastest underwater modem, leading to a 1300% performance improvement. I speak at conferences such as Europython and have an advanced degree.
Location: Singapore. Willing to relocate.
Stack: Python Data Stack(Numpy, Pandas, Scipy, Scikit-Learn), Matlab, Java.
Contact: http://shanx.us / mailshanx at yahoo dot co dot in
Get in touch with me, i'll be happy to talk to you :)