Hacker News Reader

Comment by joshe | Hacker News Reader

She was right about her book title "Rebel Talent: Why It Pays to Break the Rules at Work and in Life."

Committing fraud paid well. HBS professor salary is about $220K per year. Ten years of salary is a $2 million payoff. Big incentives to do this.

Worth a policy change by university for clawbacks of salary and especially pensions and tenure. Like insider trading she should be forced to disgorge all the profits from her fraud. Dan Ariely too.

And worth charging for criminal fraud, she defrauded her university and students. 5 years in jail seems fine.

It's time to stop pretending academics are a medieval nobles who grow faint at the idea of their sacred honor being besmirched. With huge salary opportunities for bold populist claims academics will take big risks to achieve choice jobs even if they think they will eventually be discovered.

View on HN

She also defrauded all publications she submitted fabricated science to. She defrauded all readers of her studies, and everyone that wrote about her studies. Imagine if actual public policy was made on behalf of her science.

> Worth a policy change by university for clawbacks of salary

As a matter of employment law, it's generally pretty difficult to claw back pay from an employee. And this is almost always a good thing.

Definitely a good thing. There's already channels for this. If they can prove fraud, they can go through the legal system to get that money back. It doesn't need to be a university policy, and shouldn't be - or any enforceable employer policy for that matter.

Agreed. I don't see how the current system doesn't already have channels to take action against her. If fraud can be proven, which looks like it can be, then: the school will fire her, sue her, the publisher/s can sue her, etc. Even that guy with the signed book in the comments. There is no need to update contracts to necessitate clawbacks since these essentially already exist.

I'd assume that such a policy change could actually have severe unintended consequences as it would necessitate some metric to evaluate this, which it is notoriously difficult to do with respect to research. Not to mention that most research that is wrong is not wrong to intentional acts of deception, but just that people make mistakes. I'd assume the consequences would result in this territory, making mistakes far more costly. Not something we want for research, where essentially we're trying to learn how to play the rules to a game by playing the game. Often mistakes help us learn those rules.

The university will almost certainly not sue. When does that ever happen? I've been reading about research fraud for years and am struggling to think of an example of that. It's more common that academics sue the people who reveal their fraud than universities sue academics. The US does have the ORI which mounts a handful of legal cases per year.

No, universities are a reputation economy and do their best to avoid reacting to or calling attention to fraud. They benefit from the perception that their professors are all competent and honest. This particular incident has got some press pickup due to the irony of an ethics professor who wrote a book about breaking the rules at work being caught cheating, but academic fraud is common and people who report it say that almost invariably nothing ever happens. Even in the rare cases when there's an investigation, it all gets swept under the rug. Note that:

• Harvard took over a year to request retractions, even after outsiders did the heavy lifting for them.

• This prof has silently been put on leave.

• Harvard won't comment.

• She has many coauthors on the bad papers. Nothing has happened to them. They were all blissfully ignorant.

> most research that is wrong is not wrong to intentional acts of deception, but just that people make mistakes

In reality the line is blurry to the point of being meaningless. Many glaring problems in research papers get claimed to be "mistakes" even when there's no plausible way they could be so. You have to be willing to penalize mistakes too, otherwise you can never penalize fraud.

As an example, check out the story at the top of this essay I wrote a couple of years ago:

https://blog.plan99.net/fake-science-part-i-7e9764571422

The before-surgery and after-surgery photos are clearly pixelwise identical except for the skin lesions, which were airbrushed out. Despite the sheer visual obviousness of this fraud the journal didn't notice, and when outsiders did the authors simply claimed “The photograph was taken in the same room with a similar environment; unfortunately the patient wore the same shirt” and the journal accepted this explanation! A ten year old could do better than that.

BTW don't conflate the unknown nature of answers to research questions, with the known nature of how to scientifically answer questions. The rules aren't hard, especially when you start from a base as low as "don't fake your data".

I can definitely buy this argument, but mine was about the mechanisms already existing and how the OP's comment about a policy change is needed. I think your comment just adds to this, that even if the mechanism exists that it isn't being utilized. Which results in also negating the utility of such a policy.

As to your other comments and blog, I'm well aware. And if I may add some constructive criticism, if you want to call something pixelwise identical, it helps to show a differencing map. It just makes the claim substantially stronger and clearer, especially since there's a lot of additional factors that can change pixels, making the likelihood of a match even lower. These are things that wouldn't be possible by their explanation, whereas human interpretation is more subjective (it'll be a bit messier than this though). Though even low differences are still pretty damning.

> with the known nature of how to scientifically answer questions.

I think this is a bit naive though. Generally in science we rule things out, not rule in. That's why I'd be hard pressed to say that we have a "known nature of how to scientifically answer questions" because "scientifically" is doing a lot of heavy lifting and is fuzzy. Essentially "there's wrong ways to do science, but not a clear 'right way.'" Obviously faking data isn't okay nor is just doing something and calling it an experiment. Rigor is necessary but what constitutes sufficient rigor is dependent on the strength of the claims and thus the uncertainty of those claims. A big issue we face is that we take empirical results as equivalent to analytically derived theoretical models combined with experimental results. i.e. Physics develops a theoretical mathematical model and tests experiments against that whereas {psych,medicine,ML} make a non-mathematical hypothesis and report results. The problem with the latter is that it is harder to capture uncertainty because it isn't as strictly bounded. But it is still science, and the reason this is done is because the models are likely intractable (if not provably so). But that's why Asimov wrote about the Relativity of Wrong, rather than the precise way in which we can exactly do science. In fact, if the latter was true, we'd program machines to do it. Just a bit more nuanced.

There's a pixel diff linked to from the image caption.

Point well taken. The a-theoretical nature of the social sciences (and many others) has come up before. "A problem in theory" discusses this:

https://www.nature.com/articles/s41562-018-0522-1

Oh, I missed that. In that case, yeah the evidence went from good to very strong.

As for science, my main point is that we need to embrace the noise. A lot of people try to use metrics to reduce the noise (which is good) but have this fundamental belief that noise can be absolutely removed. But this is a deep misunderstanding of what noise is. Ignoring the probabilistic nature of the universe, even a fully deterministic system has noise. A good example is the lack of absolute precision. Statistics captures error and helps us analyze that error. But error always exists due to the lack of both omniscience and omnipotence. In an nondeterministic world, it just becomes more important to embrace it. To acknowledge that it is a fundamental part of not just the model, but the thing we're modeling. In this respect, removing a noise variable doesn't just result in overconfident results, but missing a fundamental variable. Of course, noise can also be used to capture unknowns. So in any case, the noise has the be embraced. We should reduce it as much as possible, but it can't ever be removed. That's the problem: people are removing noise from their models (e.g. using a benchmark to determine evaluation absolutely)

The policies that cause 'publish or perish' to flourish are also heavily to blame here. If a scientist only is expected (or directly limited) to publish a handful of extremely good quality articles throughout their lifetime, they will likely make much more sure that the results are real.

The policies that cause 'publish or perish' to flourish

What policies are those? I thought 'publish or perish' was an expected outcome given the extreme supply/demand situation in the academic job market.

My gf is in academia (in economics) and she told me that to get a tenure you sign contracts demanding you’ll publish 4-5 relevant research papers in 5 years. A geologist friend of mine told me something similar, just with different numbers.

I agree that the up-or-out culture of tenure is bad, and so are many other terrible university policies and practices. But I would not go so far as to ascribe blame to them. 100% of the responsibility goes to the people who knowingly falsified data to advance their careers. That doesn't mean the culture shouldn't change.

> 100% of the responsibility goes to the people who knowingly falsified data to advance their careers.

I don't think anyone would argue otherwise. The issue is that if there's a culture that encourages (incentivizes, not advocates) cheating, then we should expect a culture where more people cheat. I think right now it is hard to argue that the current culture doesn't encourage cheating, and if we're being honest, doesn't have a good means of identifying it (publish or perish also decreases reviewing quality).

These policies also just make for bad science in general. In science you need people challenging the status quo, not everyone, but a significant percentage. A publish or perish paradigm encourages railroading in intellectual thought as well as encourages laziness in evaluation, as there is increased pressure on all researchers who are also the ones performing reviews. People who have zero incentive to perform a good review and many incentives to do a quick review (which is even actively encourage). This results in an over-reliance on benchmarkism, which we see in many fields. In ML we see SOTA chasing, forgetting that datasets are proxies which have their own biases and limitations as well as the same for our evaluation methods. In many of the softer sciences -- such as psychology or medicine -- we see a reliance on p-values, forgetting that 0.05 is an arbitrary value and not meaningful without additional context. All our sciences are converging on Goodhart's "Warning."

There is utility in both speed and in a slow step. The truth is that we need both. A continuum of methods and ideas. After all, we are searching in the dark and our only real advantage is parallel Monte Carlo search. The problem is that this is incredibly difficult to accurately evaluate and we are arrogantly attempting to evaluate exceptionally noisy systems with extreme precision without even having a basic model of that system. We could relate this to businesses too, but the timetables for them are typically shorter and so the chaff can more clearly be gained (often post-explained through pure intent instead of it's combination with luck). Then again, we are trying to make academia a business, trying to apply their noisy successes to a completely different model.

>The policies that cause 'publish or perish' to flourish are also heavily to blame here.

Ugh.

There's plenty of honest researchers in the field.

You don't have to be corrupt to build a nice career and, now that I think of it, only people who are equally corrupt seem to embrace/justify/tolerate these practices.

They're also expected, in a research university, to raise a constant stream of grant money. a certain subset of academics are starting to figure out that this becomes much easier when you invest a comparative pittance in a decent wardrobe , makeup , and good lighting so as to cast your research net extremely wide into populous rhetoric, Ted talks etc. The effort to reward ratio is fantastic as it invariably results in being contacted after your show, and it is a show, by people who have access to public outreach funds but very little understanding of the field nor your place in it, but want to be attached to whatever momentum you have and we'll throw money at you.

This is multiplied several-fold when you are doing this populist intellectual dinner talk circuit in one of several countries whose elite are anxious to establish (or regain) their own scientific prestige on the world academic stage and will throw unbelievable amounts of money at the most brittle and flimsy proposals so long as there is an R1 institution attached.

> It's time to stop pretending academics are a medieval nobles who grow faint at the idea of their sacred honor being besmirched. With huge salary opportunities for bold populist claims academics will take big risks to achieve choice jobs even if they think they will eventually be discovered.

Exactly. We are finally starting to see the unwinding of the 20th century scientist myth. With the advent of corporate culture in the academia, it now attracts and nurtures very different personality types.

I have a literal signed copy from her at the HBS book signing. I feel cheated and lied to.

On the plus side, you now have a book with an interesting story!

Could be she deserves an award for such a blatant violation of ethics! Sort of like Hendrix smashing his guitar.

Reply on news.ycombinator.com