HNHacker News
TopNewBestAskShowJobs

wookietrader

108 karma · joined January 3, 2012

submissionscomments
wookietrader··on Big Data vs Intelligent Data (and what Startups can do with it)
From a data analyst's perspective, let's go through what he says.

First he states something along the lines of "More data does not always help." This is right from a theoretical perspective. But: it never hurts. This is also right from a theoretical perspective, it's a result from probability theory: additional observations will always lead to less or equal variance in your estimations. There is no data like more data. There is no down side with more data.

I am not sure in what way (2) and (3) relate to big data. I'd even say that (3) is pro big data.

Then there is this term "intelligent data". Actually, I can't emphasize how badly chosen this term is. Intelligence is related to the quality of actions someone takes. Data does not take actions, It just "is". Data cannot be intelligent, just as a stone cannot be intelligent. He also thinks that data measurements should be repeatable. Guess what, in all interesting cases data measurements are not repeatable due to randomness in the source itself. One of the main challenges of data analysis is to still get robust results. He also thinks that data should be concise, e.g. that the data set at hand should be as minimal as possible to lead to the same actions. This sounds like a chicken and egg problem. How would you be able to even assess this without trying it out?

wookietrader··on I'm Not Japanese
It also introduces other problems, like latency.
wookietrader··on I'm Not Japanese
There definately is a correlation of "coming up with this thought" and "what username do I pick on HN?".
wookietrader··on I'm Not Japanese
Maybe their machine learning thinks your transsexual.
wookietrader··on German Police Used Only 85 Bullets Against People in 2011
Or you can start programming on an operating system during that period, like a certain Linus did.
wookietrader··on The Piracy Threshold
Excellent write up on that matter.
wookietrader··on NumPyPy progress report - running benchmarks
This is something like the holy grail for me.

I am a machine learning research, using the python/scipy/theano stack. One problem performance wise is that I can get my models reasonably fast with Theano--but as soon as I need a python for loop (e.g. by using Theano.scan) performance dies. As soon as you have a couple of nested (e.g. due to a Jacobian matrix instead of vector) performance is virtually gone again.

I'd love to use numpypy, and the way I see it not much is missing.

Keep up the good work! I wish I cold fund you! :/

← PreviousPage 2 of 2