Hacker News Reader

Answering a survey by pretending to be someone else is clearly unacceptable and fraud by the student if they encouraged others to do it. I’m pretty sure that would lead to that student failing the class or worse if it were discovered where I study. The same is (even more) true for faking data. That’s a disgusting violation of all scientific standards. For any somewhat larger project that might yield actual useful data our teachers are usually also involved to a degree where they would know if something like that is going on to a massive degree.

It is also standard practice to disguise these kinds of filter questions in surveys and to leave possible respondents in the dark about the ultimate purpose of the survey (insofar as that is possible and it isn’t ethically necessary to be transparent about certain aspects). That way respondents outside the filter criteria hopefully couldn’t participate even if they wanted to fake their way in. The way I do it, no one besides my advisor and a limited pool of pre-testers (who I make sure to tell not to participate in the main survey – and I also try to keep the link to the survey away from them) knows what the survey is about and what are the filter criteria.

That’s students. Most of them won’t go on to do any actual worthwhile research that will ever be cited by anyone. I would hope that the standards are even higher for research that’s published in journals.

Social research is tough even if everyone behaves ethically. No doubt about that. But I have never encountered any culture of fraudulent behaviour outside of lazy students who will never publish anything anyone cares for.

I tend to agree that defrauding data is not so much of an issue. After all, filling in a survey does not require too much effort or equipment. Cherry-picking the data and biases are the main issues with some studies.

Undergraduate students doing class assignments (or even graduate students or notionally professional researchers doing published or corporate research) using data invented to suit the desired result rather than actually following their documented methodology is by no means a feature limited to the social sciences, so I'm not sure how that is a source of particular distrust in those sciences.

It's not limited to social sciences, and I experienced this first-hand with STEM students. But the much higher scientific rigor, easier verification of results, greater separation from politics (you can't easily back your pro/anti-something beliefs by experiments with particle accelerators) and the lack of ability to just make up things as you go makes my distrust of hard fields much, much smaller than of social sciences.

While this may be largely true at the level of results of individual experiments, STEM is plagued by equally significant issues around funding. i.e. selection of which experiments will get performed. Consider the politics required to fund LHC and the necessity of there being a "God Particle" to discover. Or the reasoning applied to research in astrophysics, where at the end of a chain of tortuous logic there always should be something like "this could help us understand whether we are alone in the Universe". Or the quota of experiments needed to justify the budget for Space Shuttle. I am not objecting to these researches per se, rather highlighting that in STEM research funding at least, the Emperor has no clothes. As you say, experiments are largely sound because not easily faked, yet the distribution of research actually performed is driven by political expediencies more than it should be. Good researchers are those who understand and work within the distortions inherent to the funding landscape; a form of self-censorship.

I feel this article is relevant here: http://en.wikipedia.org/wiki/Haldane_principle

> Or the reasoning applied to research in astrophysics, where at the end of a chain of tortuous logic there always should be something like "this could help us understand whether we are alone in the Universe"

I went to the NSF grant database and searched for "astrophysics":

http://www.nsf.gov/awardsearch/simpleSearchResult?queryText=...

> Result 1. Deliverable: a terabyte-scale real-time data exchange and correlation platform. Ultimate purpose: expediate collection & analysis process for data sourced from multiple observatories.

> Result 2. Deliverable: "the coordinated observation of nearby supernovae with optical and near-infrared spectrographs on 4m and 8m class telescope..." Ultimate Purpose: The study of thermonuclear supernovae ... [which] have played a central role in the discovery of the acceleration of the expansion of the universe and have a key role in attempts to constrain the nature of dark energy.

> Result 3. Deliverable: "an underground accelerator laboratory" Ultimate Purpose: " address three long-standing fundamental problems in nuclear astrophysics: solar neutrino sources and the core metallicity of the sun; carbon-based nucleosynthesis; and neutron sources for the production of trans-iron elements in stars."

> ... that's all I care to summarize

Where is the tortuous logic and bullshit justification you keep going on about? It looks to me like the grants are going towards funding legitimate scientific inquiries into legitimate scientific questions.

There's no lack of ability to make it up as you go in the hard sciences, and there's no lack of motivation, even if it's not political, including often financial motivation.

An undergraduate's non-publishable homework assignment doesn't undermine a field's credibility. Plenty of physics students like myself bluffed lab data when we fried our experimental apparatus.

Actually, this also undermines the credibility of the institution that you attended. You can say "that's a small sample size," but it's indicative of larger problems at the institution that the honor code was not so internalized that bluffing the data would result in social ostracism.

The 'hard' sciences in the US are developing as much of a credibility problem as its 'soft' sciences. Institutional science may not be working out for us as well as has been hoped. It's a recent establishment and may not survive without a reversal in the decay of standards.

Actually, this also undermines the credibility of the institution that you attended. You can say "that's a small sample size," but it's indicative of larger problems at the institution that the honor code was not so internalized that bluffing the data would result in social ostracism.

You have fallen victim to the Typical Mind fallacy. Most undergraduate students are there for the paper that will make it easier to get a good job. Most people are not like you.

More than half of the graduate business students surveyed recently admitted to cheating at least once during the last academic year, according to a report released on Monday. The report, "Academic Dishonesty in Graduate Business Programs: Prevalence, Causes, and Proposed Action," is based on survey responses from 5,331 students at 32 graduate schools in the United States and Canada, and is scheduled for publication this month in Academy of Management Learning & Education. The survey found that 56 percent of graduate business students -- most of whom are pursuing M.B.A.'s -- had cheated, compared with 47 percent of graduate students in nonbusiness programs.

Please note almost half of nonbusiness GRAD programs cheated. It is going to be way, way higher in undergrad.

>The 'hard' sciences in the US are developing as much of a credibility problem as its 'soft' sciences. Institutional science may not be working out for us as well as has been hoped. It's a recent establishment and may not survive without a reversal in the decay of standards.

Key words: in the US. You can't generalize from one institution in one country to all institutions on the planet.

I have two comments with respect to what you wrote:

FWIW I'm an undergrad who is currently writing his thesis (empirical economics). I use the German Socio Economic Panel data set, which is a representative survey of ~11,000 German households (totaling about 79,000 individuals over 28 years). The reason why I picked a data set instead of collecting my own data is precisely to avoid the situation of the social sciences student you describe. As an undergrad, getting data is extremely difficult. You usually get about 6 weeks to write your thesis and you have absolutely no training in data collection and/or survey design. Getting data from students is your best bet, but nobody wants to answer your questionnaire because…hold your breath…everyone is sending out questionnaires. I get spammed with these things. Luckily, nobody has asked me to pretend that I am a 13 year old. So quite frankly most students panic after about 2-3 weeks and they will try to do anything to get out the situation. The alternative is not graduating and paying tuition for another year.

Secondly: Undergrad thesis =\= scientific paper. The fraction of undergrads who end up being social scientists is…tiny. Out of 300-400 econ students who graduate at my uni every year, I think 20 or less will go onto grad school (I'm proxying the number by thinking about how many went to do an MPhil). Now econ grad school is a different beast. Essentially you either end up doing experimental research or you work with observational data (assuming you are not a theoretician, those exist too; Economics is pretty much applied mathematics at that level.). Experiments can be checked and working with observational data requires a trove of econometrics. Trying to fool a panel of econometricians isn't easy, especially when these days you have to send in your data to the journal (or so my supervisor says; he has a paper forthcoming in the Quarterly Journal of Economics).

Social science in general is becoming more and more computational and statistically rigorous. Gone are the days when an "expert" would find a correlation between two variables and then speculate which way the causality flows…if any.

So you're, on a sample size of one or two undergrads, concluding that a whole science field is "nonsense"?

No, they are trusting the statements of another graduate student who had a much larger sample size.

Also by "things like these" I implicitly meant that I've heard many stories like that, including long discussions here on HN about invalid use of statistical methods. I just wanted to provide a single example situation that coincidentally happened just few minutes before I found this submission.

That is pretty shoddy, but bear in mind this is an undergrad thesis, and so (apart from teaching very bad ethics to future researchers) this has no bearing on research that will ultimately be published.

As an aside, I first misread your post and thought you were describing the process of taking your own survey, while putting your self in the mind-frame of the person filling it out, in order to imagine what goes through someone's head when they fill it in. This seems like an extremely good idea.

While that is of course fraudulent and would have meant failing the thesis where I studied (education, the softest of soft sciences ;)), a bachelor's thesis has absolutly nothing to do with the scientific field - a bachelor's degree isn't even enough for getting a job in the real world, after all.

Science starts at dissertations.

That's one of the most stupid comments I've seen you make, in what, three years on this site.

Thankfully, I don't judge you entirely on it ;)

Thanks for your honest feedback :).

Could you please elaborate a little bit on what you consider stupid in it? I'm always eager to learn :).

Not OP , but your story isn't even anecdotal really it verges onto a friends of a friend story.

Plus rather than saying undergraduate thesis say assignment. Because that's what it really is, not at all related to peer review papers or a doctorate.

I think you're speaking from a position of ignorance, and discounting an entire field based on your biases, with confirmation in the form of an anecdote about a grad student (or undergrad? - it's not clear).

Of course my comment is partly tongue in cheek, given that its form necessarily shadows your own statement for my desired level of irony.

My own biases come in part from my GF. She's an economist - a social science - and has opened my eyes in many respects. Most of all, it's extremely hard to get good data in the social sciences. She works in healthcare policy evaluation, but all the people collecting data are not trained in it - they're filling out forms, themselves often designed by people not trained in it either. And of course she runs up against the usual "you have to do it for the children" people who dismiss any kind of rational analysis of the cost / benefit of various different healthcare interventions.

Anyway, I'm a bit sensitive about this, because I see a lot of arrogant ignorance, especially in IT, with guys (usually guys in our field) dismissing whole fields that very clever men and women have dedicated entire lives to, without bothering to learn anything about what they're dismissing.

I saw the analogy in some book: imagine there is a shoemaker, who has no access to good materials and good tools. The result of course is that his shoes are terrible. And why you can understand all of the reasons why shoes are terrible—they are still terrible. Good data is hard to get. Yup. Or do you think the data on Higgs boson was easy to get?

That's a different kind of hard to get.

Bad data is easy to get.