Comment by mseebach | Hacker News Reader

Comment by mseebach | Hacker News Reader

If you analyse reams of statistical data for patterns, it's really unsurprising that, after the fact, a few patterns (sewer system, traffic, crime rates and AIDS cases) cherry picked for the fact that they match well .. well, matches very well. In other words, it's not surprising that large data sets have patterns. What would be surprising would be if a very large number of variables were predictable, but that does not seem to be the case.

There's a section in The Black Swan devoted to the fallacy that we tend to project the fact that we understand the rather narrow field of physics very well - to the extend where we can predict things in that domain reliably - to an expectation that we can do the same for other fields, when in fact, it's physics that's the outlier in our ability to understand it so well.

In the end all his efforts seems to come down to the conclusion that cities exists because they provide economies of scale. I couldn't provide statistical proof for that fact to save my life, but still .. well, duh?

As West is a highly trained physicist, I'll believe he's a bit more statistically careful than that. In the most crude, there's a thing called the Bonferroni correction which helps to counteract cherry picking effects. Beyond that there are a great deal of other methods to account for overconfidence.

If we were not able to identify a small number of relevant patterns in a large sea of options, how would we learn anything?

If you've got a mountain of data and are willing to run a lot of models you can indeed identify ones with good tradeoff between prediction and generalization errors. It's very possible that some of your models are weaker than you expect due to cherry picking, but it's even more possible you've identified some useful trends.

As you said, he seems to be restating very common knowledge about economies of scale. Since he's not saying anything terribly surprising, I don't think there's much need to get up in arms about multiple comparisons.

Moreover he's looking for the on-average rates of growth of these economies of scale and identifying outliers which will likely be interesting. I think it's pretty exciting research.

I couldn't provide statistical proof for that fact to save my life, but still .. well, duh?

There's an enormous difference between saying "I think X is true," and "I have significant evidence based on observation and theory to support the conclusion that X is true."

Also: http://lesswrong.com/lw/im/hindsight_devalues_science/

“There are always going to be people who say, ‘What about the crayfish?’ ” he says. “Well, what about it? Every fundamental law has exceptions. But you still need the law or else all you have is observations that don’t make sense. And that’s not science. That’s just taking notes.”

I'm surprised he used a biological explanation, which I find to be faulty. I would have guessed that the growth and evolution of a city is more like that of a star. Every city lies somewhere along a "hertszprung-russell" diagram of urbanity.

He works with the Santa Fe Institute that specializes in studying complex adaptive systems. What all CAS have in common is the organic growth patterns that seem endemic to biological organisms. Also, if you consider the fundamental building block of the city to be its people rather than the physical infrastructure, then you are dealing with a biological system. This is no different than looking at an ant colony as a superorganism - yeah they have some dirt and tunnels and some rocks, but for the most part the important factor in the colony is the mass of individual ants.

Interesting point, which brings up a pseudo-philosophical question, should we consider humans living in large cities a kind of super-organism in a similar manner?

Reply on news.ycombinator.com