Hacker News Reader

Yes, if anything we need better documentation of methods. This is something often overlooked with the open-scientific-data movement also: data on its own is not very useful (in the sense of being insufficient to draw valid conclusions) if you don't have good documentation of how precisely it was collected, which is critical to interpreting it correctly... otherwise you just have some numbers in a CSV file. Traditionally PhD theses are one of the few places that really exhaustively documents what exactly they did and how, while papers tend to gloss over a lot of things. There's been a little noise about that [1], but not a lot of movement so far.

I do think a movement of theses away from monograph-style to collection-of-articles style is okay, though.

[1] http://scholarlykitchen.sspnet.org/2014/11/18/nevermind-the-...

My thesis was very much a recipe for how to make my experiment work. We frequently used the dissertations from former students as reference manuals for their methods. I cranked mine out in 3 months.

Amusing anecdote: This was 20+ years ago, when technical word processing was still a work in progress. Many of my peers struggled endlessly with formatting. My plan was to create a mechanically rough manuscript, get through the defense, then worry about formatting afterwards. It worked, with a hitch: I never got page numbering to work, and ended up pasting the page numbers in by hand. Also, many of the figures are obviously pencil drawings that were photocopied.

There is no need these days for brevity with methods since everything is online. Put it all into the SI and be done with it.

One problem with many methods is the lack of description of the pitfalls and critical steps. When you go to repeat an experiment you often don't have exactly the same equipment or reagents as the original paper. Most of the time this does not matter, but in an annoying number of cases some particular step needs to be performed using a precise piece of equipment or brand of reagent. It is only when you can't get the experiment to work and you contact the person on the bench who did the research that you learn "yeh we tried all the other reagents and only brand x works". If there is something critical put it in the method that it is critical!

In my experience a lot of very critical steps in experiments are frequently never formally defined in literature or even within lab notebooks, existing only as institutional knowledge. What may seem like an obvious step not worth docimenting to the original investigators is non-obvious to people looking to repeat the work, leading to different results.

Good comment. My addendum to follow is not directed to you, but to non-scientific readership.

Yea, it's boring for professor on the thesis committee to read, but this sort of stuff is very valuable for students and postdocs, who often struggle to reproduce poorly-documented results from other labs.

There's both absolute and relative poor documentation. The brevity of most scientific publishing (coupled with the lack of any sort of external incentive for clarity, as you point out) means that even papers that are good, by the standard of scientific papers, can be a real bear to turn into working examples. I personally feel that a lot of what makes various labs good is knowledge of the things that aren't going into the papers. A dissertation, with (potentially) a lot more room to go into details, can serve both as a source of this information for external readers, as well as a convenient internal reference for people in the lab who need to dig up some details of their prehistory.

edit: to clarify, my perspective on this is coming from physics. In other fields, there's more of an emphasis on explicit protocols and "methods sections" that presumably help with this sort of thing.

http://tinytocs.org/

Agreed! Not every thesis is great, but most research isn't great either. Good students write good theses and I've found many with whole chapters on important topics that are just glossed over in the research papers.

Maybe someone should do a PhD to discover if the Tweesis is the future.

>Ahh yes, just what we need. Thesis-by-tweet...

Look, a lot of things don't need 100 pages of text. Of course, as a researcher, what you don't want to do is to make your research easy to read, because people may then assume that it was easy to do and therefore not impressive.

> As a researcher, what you don't want to do is to make your research easy to read

Mmmh.. Disagree. This is the same idea as those software that nobody wants to explain you what does exactly, and therefore don't get a lot of customers using it after the novelty wears off. This is a red flag for mediocrity for me. Of course you should aim to get so many readers as possible.

If you're simple, clear and first, you're going to get plenty of citations, though.

For CS? Not in your thesis, but maybe in a paper based on the thesis.

In natural sciences, thesis is based on papers. In CS, paper is based on thesis.

Both actually. Its just that your thesis won't get many if any non-self citations, even if it is really good. People will cite your paper, which they bother to take time reading (hopefully).

The conundrum during all research talks.

No, just add complicated-looking figures and you're fine.