Comment by bjaress | Hacker News Reader

Comment by bjaress | Hacker News Reader

Parent Full thread

I think computer models just mean you do replication the other way around. Of course anyone who runs the same program gets the same results, so ordinary replication is pointless--instead you answer the same question with a totally different program and see if your answer is close.

And then, of course, you wait and see if the predictions of the models come true. You can't reset the world and re-test it, but you can re-run the models and ask for more prediction in the future and wait some more.

Climate scientists do both of those things all the time because they're in one of the most heavily scrutinized fields.

Isolating variables just means you compare two setups with everything the same except one. That's actually one of the things the models are for. You can't re-run the world without humans, but you can re-run the model with humans turned off.

Then someone else can do the same with their totally different model. And if both of your answers match reality with humans on and each other with humans off, well maybe the difference between humans on and humans off is the impact of humans. Or maybe not. But adding more different models helps.

In short: climate science generates testable hypotheses, does replication, and isolates variables. It's possible they're wrong (and publishing source code is a good idea) but they don't have a methodology problem. And they're probably right.

Also, Michael Crichton basically wrote Hollywood scripts in novel form. He's not a good source on anything.

> Of course anyone who runs the same program gets the same results, so ordinary replication is pointless

That would be true, if anyone actually distributed their actual code. Pick a journal article at random in any field that describes results from a computational model, and 99% of the paper will describe the results and not the model. The paper will never contain the complete code (which is fair enough, since it would be too long); 1 paper out of 100 will have excerpts of the code, and another 10/100 will have a URL that claims to have the code. If you actually follow that link, you'll find that 2-3 times out of 10 the code won't actually compile or run, and 9 times out of 10, the figures in the paper were generated by tweeking some parameters not defined in the paper whose particular values the author never recorded, and so even the author couldn't reproduce what he actually published, even if he wanted to.

Some people are trying to make institutional change here, see http://www-stat.stanford.edu/~wavelab/Wavelab_850/wavelab.pd...

But really it's a pretty sad state of affairs.

> He's not a good source on anything.

Perhaps you mean this in the sense that we should trust in science and not in authority? Because as hack writers go, his academic qualifications are better than most. He's an intelligent guy, and it seems fair to give at least some weight to his opinion. Equally, it seems unfair to presume a priori that he's not a good source of information. At the least, you'd should argue "He's looney about X, which we all agree is false, therefore we should not trust him on Y". Simply saying "He's looney about X" doesn't add much information and merely pits your authority against his.

I agree with your summary of computer models, and your basic judgement of climate scientists, but fear that many of the influential climate science papers don't adhere to this standard. Frequently the technique is to tweak the parameters of a number of models until each creates "realistic" results, generate a small number (1-3) of simulations with each model, and then create an unweighted average of this ensemble so that the high and low estimates cancel. The meaning of this is much harder to interpret than the case where all the models predict a similar outcome with identical inputs.

http://www.ipcc.ch/ipccreports/tar/wg1/347.htm#tab91

----

CRICHTON, (John) Michael. American. Born in Chicago, Illinois, October 23, 1942. Died in Los Angeles, November 4, 2008. Educated at Harvard University, Cambridge, Massachusetts, A.B. (summa cum laude) 1964 (Phi Beta Kappa). Henry Russell Shaw Travelling Fellow, 1964-65. Visiting Lecturer in Anthropology at Cambridge University, England, 1965. Graduated Harvard Medical School, M.D. 1969; post-doctoral fellow at the Salk Institute for Biological Sciences, La Jolla, California 1969-1970. Visiting Writer, Massachusetts Institute of Technology, 1988.

http://www.michaelcrichton.net/aboutmichaelcrichton-biograph...

Crichton's scientific standards can't have been THAT high, considering how he got hoodwincked into believing Jack Houck's whole spoon bending spiel. I've attended one of Houck's parties and the whole thing was quite remarkable, not for the spoon bending but for the gullability of 90% of the people there.

That certainly could be a legitimate case of "He's looney about X therefore we shouldn't trust him about Y", but I'm not familiar with either Houck or Crichton's beliefs. The quote I can find from him on it seems within reason: 'I think that spoon bending is not "psychic" or bugga-bugga. It's something pretty normal, but we don't understand it. So we deny its existence.'

On the other hand, he also says "More than seeing adults bend spoons (they might be using brute force to do it, although if you believe that I suggest you try, with your bare hands, to bend a decent-weight spoon from the tip of the bowl back to the handle. I think you'd need a vise.)"

http://www.crichton-official.com/qa-travels.html

I just grabbed a spoon and tried it. As expected, contra Crichton, I had no trouble bending the handle to touch the bowl -- no vice required. And no trouble twisting it 360 after bending. But it would be a little surprising that one could exert that much force without noticing. And there was some interesting annealing and tempering going on: it was much harder to untwist than to twist, easier to unbend than to bend, and subsequent bends preferred new locations to repeat bending. So the scale tips a little toward looney, but I'd have to read more before discounting him. And I'm willing to believe there might be some metallurgical property worth exploring here, although I'm pretty sure it has nothing to do with telekinetics or psychics.

But you've actually been to a spoon bending party, and I haven't. Do you have a loonier link?

Just google Jack Houck, you'll find plenty.

Funny anecdote - I honestly tried to use his method of 'feeling the metal get soft and then quickly use this moment to bend the spoon'. I didn't feel anything, so after seeing everyone around me get into some kind of ecstasy, I decided to actually bend the spoon to get an idea of how hard it was. It wasn't hard at all! (Just get some low quality, cheap spoons and forks, they're very easy to bend.) Now, when Jack Houck came around, I showed him my spoon with a sad face and told him it hadn't worked for me, and I had just 'used my muscles' to bend it. He took a few moments to examine it, then proclaimed that he could see in the metal that it had actually melted, that there were features inconsistent with 'cold bending' and that I had very great mindpower but just didn't realize it.

The crowd at this party was very much into new age stuff, crystal healing and all that. In fact, Jack Houck was doing a seminar the next day to teach people healing powers using the same 'energy' that was used to bend spoons, which he had come to consider as a party trick of little interest compared to the healing powers.

Reply on news.ycombinator.com