Hacker News Reader

Comment by oglop | Hacker News Reader

“A recent study by economists at the Federal Reserve found that less than half of the published papers they examined could be replicated, even when given help from the original authors. Mr Levitt’s counterintuitive results have fallen out of fashion and economists in general have become more sceptical.”

Didn’t realize replication was so bad in Econ too.

View on HN

Depends on which part of econ. Micro and econometrics fare pretty well (there was even an econ pseudo-Nobel awarded to Vernon Smith for figuring out how to do reproducible experiments in micro), but macro is the darkest of swamps.

If the purpose of half of economics study was to justify political decisions, rather than to learn facts about the world, that replication rate would be what you'd expect.

> If the purpose of half of economics study was to justify political decisions

I mean, let’s be honest, is it _not_ actually this? Maybe economics should just give up the charade and accept its role as a political tool and not something with a scientific base so comically bad, even the softest of other sciences look positively mathematical in comparison.

There are lots of good reasons why you can't admit that it's a con, most of which are some variation on "it's good to lie."

If you want to use propaganda but you tell people your propaganda is propaganda, it won't work.

Which specific social sciences do you believe are doing better?

None of them I would guess.

Math is math and the hard sciences like physics, chemistry, biology,and engineering are all good assuming somebody isn't mining p-values or something like that.

The social sciences require a ton of data and simulating something like the economy with trillions of variables and bounded rationality just isn't possible. If you wanted to simulate an agent like myself you'd likely need millions of lines of probabilistic code and then you'd need millions of other agents running on a supercomputer the size of Rhode Island.

Biomed is already too complex to simulate fully, which is why simulating is just, not really a thing that is done to find new knowledge. Heck, some quantum interactions are already (so far) computationally intractable, which is why quantum chemistry exists as a field at all, it's far from trivial. That doesn't mean the best practices developed for medicine won't help in the social sciences as well.

The issue with degrees of freedom can be mostly pinned down with preregistration, among other measures. It won't solve the problem of having more type I errors at any specific error rate the more science you do, but it does give assurances the target type I error rate is accurate.

Econ is a notorious "nothing and everything topic" (not being fair here, but economists also talk about this). It doesn't really fit anywhere. It's not very practical to study scientifically.

I don't want to insult Sociologists here, but economics can be very much like sociology for money issues.

It's intuitive that replication would be terrible in economics (not taking a shot at you for not "realizing"; we don't think about every subject in the world).

What you really want to do is take one society, run it for 5-200 years. Then go back in time, take the same society, and try again with one small change. In practice, you might need to repeat many times because you would expect chance to change such a complicated system (butterfly effect).

"In practice"... listen to me, I'm being absurd. This is clearly impossible. Of course there's ways to try anyway, but they're all known to be "bad science".

It's bad. But the field has moved towards causal inference methods anyhow, and away from the "cutenomics" Freakonomics championed.

I mean half went be bad but like 10-20% would be the expected if it matched the other sciences

Reply on news.ycombinator.com