HNHacker News
TopNewBestAskShowJobs

jeroenjanssens

837 karma · joined November 12, 2012

https://jeroenjanssens.com
submissionscomments
jeroenjanssens··on Data Science at the Command Line
Believe it or not, it's partly thanks to you, HN, that I wrote this book in 2014 in the first place [1]! It's humbling to see it listed here again now that the text has become available under a CC BY-ND license. Thank you for your help in spreading the word.

All this attention (read: likes, shares, and page views) is making me wonder whether it's worthwhile to write an update (or even a second edition). What do you think? What would you like to see changed or added?

[1] https://news.ycombinator.com/item?id=6412190

jeroenjanssens··on Introduction to R Programming
I can't believe you start counting at 0 ;)
jeroenjanssens··on Introduction to R Programming
The book "R for Data Science" by Garrett Grolemund and Hadley Wickham (O'Reilly, 2017) [1] provides a comprehensive introduction to modern R and a set of packages known as the tidyverse. Highly recommended.

[1] http://r4ds.had.co.nz/

jeroenjanssens··on Exa, a modern replacement for ls
Sometimes the Unix philosophy has to be taken with a grain of salt.
jeroenjanssens··on Uber Founder Travis Kalanick Resigns as C.E.O.
Python 2? Absolutely barbaric.
jeroenjanssens··on Show HN: Nice-looking Python bindings to V8
In Dutch we call this "plaatsvervangende schaamte".
jeroenjanssens··on Linux utils that you might not know

    $ curl -s http://datascienceatthecommandline.com | pup '.sect3 > h3 text{}' | head
    alias
    awk
    aws
    bash
    bc
    bigmler
    body
    cat
    cd
    chmod
jeroenjanssens··on Learn the basics of data science with these books
Thanks! I must admit that the thought has crossed my mind. However, these days all my time is spent consulting and giving training so I can't make any promises.
jeroenjanssens··on Learn the basics of data science with these books
I'm flattered to see Data Science at the Command Line next to these great titles, but I'm not sure if I would recommend it to learn the basics of data science.

DSATCL discusses the ideas behind various cleaning and visualization approaches and several machine algorithms, but only briefly. My personal recommendation would be to first gain some experience with these topics using Python and/or R. If you're afterwards curious to find out how the Unix command line can help to do data science, well, then there's only one book I can think of! ;)

jeroenjanssens··on Gnuplotting – Create scientific plots using gnuplot
If you're using LaTeX anyway, then you might be interested in TikZ [1]. This repo [2] contains some of the figure I created for my thesis.

[1] http://www.texample.net/tikz/examples/feature/plotting/

[2] https://github.com/jeroenjanssens/phd-thesis

jeroenjanssens··on The NumPy array: a structure for efficient numerical computation (2011) [pdf]
What an interesting juxtaposition of idioms :)
jeroenjanssens··on Ask HN: Freelancer? Seeking freelancer? (May 2016)
SEEKING WORK - Remote

As an independent data science consultant and trainer, I help organizations making sense of their data. Additionally, for one day a week, I'm an assistant professor of data science at Tilburg University. Previously, I was a data scientist at Elsevier in Amsterdam and startups YPlan and Outbrain in New York City. I have an MSc in artificial intelligence from Maastricht University and a PhD in machine learning from Tilburg University. Recently, I wrote a book titled Data Science at the Command Line, which was published by O'Reilly Media. My website (http://jeroenjanssens.com) provides a more detailed overview of my background.

I have experience and affinity with: analyzing data using R and the Python scientific stack; organizing workshops and meetups; writing and presenting for technical audiences; managing data in MySQL, MongoDB, Redis, and AWS S3; collaborating with international teams of engineers and data scientists; programming in Python, JavaScript, and Bash; visualizing data with ggplot, TikZ, and D3; and performing large, distributed experiments.

I'm currently available for three to four days a week. Simply drop me an email at datascience@jeroenjanssens.com and we'll take it from there. Looking forward to it!

jeroenjanssens··on Learn the Command Line
Awesome! I wrote something related: http://datascienceatthecommandline.com
jeroenjanssens··on Jq – A Command-line JSON processor
I have found `xmlstarlet` to be really useful for working with XML. You can achieve a lot with some basic XPath expressions. No need to learn XSLT. (I should actually update my book Data Science at the Command Line to include this tool.)
jeroenjanssens··on Tapkee, a dimension reduction library
Comes with a great command-line tool.
jeroenjanssens··on IBash Notebook
Hence the interrobang in the title. I hope that someone smarter than me can alleviate those two issues.
jeroenjanssens··on Command-line tools can be faster than your Hadoop cluster
Author of Data Science at the Command Line here. Thanks for the nice blog post and for mentioning my book here. While we're talking about the subject of education, allow me to shamelessly promote a two-day workshop that I'll be giving next month in London: http://datascienceatthecommandline.com/#workshop
jeroenjanssens··on One Thing Well – A weblog about simple, useful software
Related: http://datascienceatthecommandline.com/#tools
jeroenjanssens··on sshrc – make your ssh sessions feel like home
As a complement to this nice tool, I can recommend sshfs [1], which enables you to mount a remote machine as a local folder.

[1] https://www.digitalocean.com/community/tutorials/how-to-use-...

jeroenjanssens··on Useful Unix commands for exploring data
First blog post was the inspiration for a book, which is almost finished: http://datascienceatthecommandline.com
jeroenjanssens··on Useful Unix commands for exploring data
Try `body`: https://github.com/jeroenjanssens/data-science-at-the-comman...

    $ echo 'header\ne\nd\na\nb\nc\nb' | body sort | body uniq
    header
    a
    b
    c
    d
    e
jeroenjanssens··on Ask HN: What's your favorite command-line tool for working with data?
I have played a bit with XMLStarlet [1] and xml-coreutils [2], but not too much. Let me know how you find them!

[1] http://xmlstar.sourceforge.net

[2] http://xml-coreutils.sourceforge.net

jeroenjanssens··on Jq: A lightweight and flexible command-line JSON processor
I'd just use xml2json and then apply jq. Usually gets the job done. The benefit of that is that you only need to learn jq.

See this blog post I wrote a while ago for an example: http://jeroenjanssens.com/2013/09/19/seven-command-line-tool...

jeroenjanssens··on Data Science Toolbox
Anaconda is an easy-to-install distribution that contains many many Python packages. It is not a virtual machine.

The Data Science Toolbox also contains Python and the most popular packages for doing data science. Besides that, it also contains R and, again, some great packages.

The main difference, though, is that the DST aims to be a platform where authors and teachers can create custom software and data bundles for their readers and students. (I have been thinking about creating a bundle that would install Anaconda on the DST.)

If you only need to use Python, then Anaconda may very well satisfy your needs. Good luck!

jeroenjanssens··on Data Science Toolbox
That's a nice coincidence! (I can imagine that all these names can be confusing.) The Data Science Toolkit focuses on lots of interesting APIs. This blog post [1] provides a brief comparison.

[1] http://jeroenjanssens.com/2013/12/07/lean-mean-data-science-...

jeroenjanssens··on Data Science Toolbox
Author here. Although the software contained in the base is indeed relatively easy to install, you'd be surprised how often people struggle installing Python and its packages on Mac OS X or Microsoft Windows!

The initial reason I created DST was that I'm currently writing a book called Data Science at the Command Line. Installing the command-line tools discussed in this book is unfortunately not straightforward. I wanted to offer potential readers a Vagrant box (with a shell provisioner) that installs these command-line tools. Similar to what Matthew Russell has done for his book Mining the Social Web [1]. This blog post [2] provides some more explanation and compares four different solutions.

The software packages that you mention sure are exciting! If there is an author or teacher that wants to teach these to her readers or students, then we should create a bundle (i.e., Ansible playbook) for them!

[1] https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-Ed... [2] http://jeroenjanssens.com/2013/12/07/lean-mean-data-science-...

jeroenjanssens··on Lean, mean data science machine
Thanks! I completely agree with you on making the analysis reproducible. IPython Notebook is a wonderful tool, and I use it a lot myself. (You can even run shell commands from IPython.) However, using the command-line doesn't necessarily mean that the analysis will be difficult to follow or maintain. On the contrary, because you're making use of existing tools (and perhaps even create new ones), you are processing data on a higher level, which can actually make it easier to read and maintain.
jeroenjanssens··on Explain Shell
See https://github.com/jeroenjanssens/data-science-toolbox/blob/... for a bash script that interfaces with explainshell.com.

I wrote this script a while ago. I just needed to update the request because the API had changed a bit since then. Please note that it depends on the scrape tool in the same repository, which in turn depends on the python packages lxml and cssselect. But once you have that set up, you can explain commands from the shell! :-)

jeroenjanssens··on Detecting anomalous senators
Thanks! I hope you understand that I had to be brief in the blog post. Since you seem so interested, I recommend you read the technical report. Do let me know when you've got any questions after that. If you want, you could even clone the SOS repository and change the Python code to test out your hypothesis!
jeroenjanssens··on Detecting anomalous senators
The brief explanation of SOS (which starts at the top of the blog post) uses a toy dataset. This toy dataset has 6 data points. Each data point has two features. In other words, we have 6 data points in a 2-dimensional space.

As for the roll call voting data, the dataset has 103 data points, where each data point has 172 features. The outlier probability of a senator depends on the location of its corresponding data point in the 172-dimensional space.

← PreviousPage 2 of 3Next →