erikbern

let me fix

(author here) not sure what happened, but the site is up again.

> Statistical power is simply the likelihood that the difference you’ve detected during your experiment actually reflects a difference in the real world.

This seems incorrect to me. Isn't statistical power the likelihood that the null hypothesis would generate an outcome at least as extreme as what you observed?

I'm guessing the issue has a lot more to do with peeking at the outcome and not correcting for it (and similarly running many tests)

http://www.stat.columbia.edu/~gelman/research/unpublished/p_...

This assumes there's a fixed demand for legal help. You might as well envision that the efficiency gains cause a huge drop in legal costs which unlocks a huge latent demand. For instance software engineers are probably 10x more efficient now compared to 20 years ago, but this does not mean mass unemployment.

I think the nan issue you are referring to was caused by some weird stuff with Bazel.

I put together an AMI in virginia: ami-cf5028a5 and if you have a masochistic streak, here are the steps to do it yourself: https://gist.github.com/erikbern/78ba519b97b440e10640

The main issue I'm still seeing is that g2.8xlarge with my AMI doesn't run 4x faster than g2.2xlarge even if correctly detects the 4 GPU's. Haven't had time to identify the issue though.

I doubt this has much to do with traffic. It can be explained by another observation: on average you spend more time in slow lanes.

The checkout at a supermarket has the same phenomenon. Let's say with 50% probability you choose a line that takes 10 minutes and with 50% probability you choose a line that takes 20 minutes. On average now 67% of the time the other line will move faster.

I think that's pretty useful approach. It's kind of similar to Hinton's latest work on model compression: http://www.ttic.edu/dl/dark14.pdf

The problem with deep models is when you end up having more than 1 hidden layers, you have a big matrix multiplication to get between the layers. If your hidden layers are a few thousand units, that's still pretty slow. Doing things in minibatches or on the GPU speeds it up significantly, but I'm guessing it's still orders of magnitudes slower than whatever Stockfish uses

Thanks for the comment. I'm definitely happy to admit I have very limited understanding of computer chess and I think the likelihood of stumbling upon a new approach to chess engines is close to zero.

That being said, I'm not convinced there's anything "magic" about the fast deep search. There's a broad trend of big complex hand-tuned models being replaced by simple models with more parameters and many many orders of magnitude more training data (eg. machine translation, computer vision). There's probably a lot of domains where this approach doesn't work though. Maybe chess is one of those applications. We don't really know until we try :)

Sunfish is really impressive work. From my (brief) understanding of Sunfish, the evaluation function is essentially equivalent to a hardcoded 1 layer network in Deep Pink.

You're right that everything else equal, a better evaluation function should lead to a better chess engine. However in practice I think better evaluation functions means slower evaluation function. So there's some really interesting trade-off there. I doubt humans evaluate more than a few thousand positions, so it seems like a slow but more accurate evaluation function could play chess pretty well

We've built Luigi at Spotify to solve a lot of similar problems: https://github.com/spotify/luigi

Might be worth checking out if you are building large data flows. We probably run 10k Luigi "tasks" every day, of which the majority is Hadoop jobs. They are all organized in a large dependency graph expressed within Python, and you also get visualization, exception handling, atomic file operation, etc.