Hacker News Reader

Comment by wheaties | Hacker News Reader

This is one of the reasons I went into math and software. In math there is only black and white, provably true and patently wrong. Still, part of the reason this is so is that to show any results everything must be laid on the table. CS and Math papers are therefore open by design and the world has benefited. I hope the rest of science will move in this direction.

View on HN

I'm guessing you have not read Proofs and Refutations by Imre Lakatos. It is worthwhile.

Also if you read on the history of mathematics, the various back and forth foundational issues raised by infinitesmals, Fourier series, proof by contradiction and the like are very interesting. Sure, it is possible for a modern mathematician to look at all that and say, They just didn't know how to do it, HERE you go. But over decades and centuries people were somehow successfully continuing to do math, even though they knew that something serious was wrong in their understanding.

At least those historical mathematicians acknowledged their gaps of knowledge. Cancer research is a far cry different.

You clearly are unfamiliar with the history.

If you can find a copy, George Berkeley's The Analyst was a cogent criticism of the foundations of calculus in his day. (He didn't have answers, but he identified real problems.) A variety of books were written by mathematicians in response. Uniformly these were much lower quality, and consisted of defenses of the foundations of mathematics, rather than acknowledgements that there were real issues. His criticisms were not taken seriously until that mathematical framework completely fell apart when Fourier series constructed "obviously impossible" things.

Today we are used to a very general notion of a "function". But historically a square wave simply wasn't a function. (Because if it was, how did you know how it interacted with infinitesmals?) And getting one out of the sum of a bunch of well behaved sins, with some simple integration, was a huge shock.

Yikes! Thanks for setting me right.

I should have said some historical mathematicians then.

There are 'proofs' that contain errors, just as there are science papers that reach incorrect conclusions.

Also, it's not necessarily easier to verify a proof than it is to repeat an experiment (or perform additional supporting experiments), especially for complex proofs which very few people know enough to completely understand (eg. of Fermat's last theorem, or the Poincare conjecture).

CS has similar problem, specially in AI fields. Researches often claim the accuracy rate more than 95%, but in reality that is much lesser (in fact less than 10% in most of the claims).

Yes, that's why I no longer consider pursuing PhD in CS. It's a waste of life to reproduce non-reproducible papers that is full of buzzword.

Hopefully, more journals and confs will follow the Open Access and Reproducible Research model like IPOL[1].

> Each article contains a text describing an algorithm andsource code, with an online demonstration facility and an archive of online experiments. The text and source code are peer-reviewed and the demonstration is controlled.

[1]http://www.ipol.im/

Having worked in computer vision and robotics, I can attest to this.

Yes. Of the fields I've implemented algorithms from papers in, computer vision seems to be particularly adverse to discussing the (often critical) downsides of their algorithms. Often its something like "Camera calibration isn't exactly perfect? Well this scene reconstruction technique simply wont work."

Not to be too picky (am a mathematician too) but you ought to read about the history of the theory of "limit cycles" of planar vector fields and Dulac's "Theorem". Quite a lesson.

Edit: sorry, could not help writing this later (was on my iPhone before). The thing is that Dulac 'solved' an important part of one of Hilbert's problems. Was a 'proof' for about 60 years.

Pick a paper on NLP. Measures of document relevance and summary quality are not black and white, especially when you account for the ways the authors munged their test sets and scores.

> In math there is only black and white, provably true and patently wrong.

You'd probably be interested in Godel's incompleteness theorem.

Also, exact reproduction of other people's experimental work is highly unusual in CS, despite the theoretical possibility. Usually the materials and methods sections of papers in hard sciences like physics, chemistry, and biology are much more detailed, to the point of being recipes.

Finally, CS is not really a science. It lies somewhere between math and engineering, which are also not sciences.

If that were an actual possibility in other fields everyone would be delighted. Sadly, biology (human minds and societies as well) are much too complicated for this to work.

This is an actually challenging problem. I don’t think running away from it and resorting to cheap tribalism is a great idea.

Reply on news.ycombinator.com