Hacker News Reader

Comment by opticfluorine | Hacker News Reader

While pursuing my Ph.D. in chemistry, I was asked to peer review an inorganic chemistry publication in which the author attempted to fake crystallography data. In this case the telltale sign was that they left the hydrogen atoms in the structure - X-ray diffraction is unable to resolve the hydrogens, you need a more sophisticated technique like neutron diffraction for that.

What it turned out they had done is optimize a predicted structure of their molecule using density functional theory (DFT), then draw it like a crystal structure would be displayed (a 3D plot with error ellipsoids at each atom, with probable bonds shown) and tried to pass it off as a true crystal structure. My advisor missed this when he reviewed it first - if you're reading quickly, you just see a crystal structure and accept it for what it is. So I wonder how many like this slip through the cracks every year. Apparently a lot, if the CCDC identified ca. 1000 of them.

View on HN

Does this not immediately destroy one's scientific career and invalidate all past papers? It seems like it should.

Maybe there should be a bounty on uncovering these?

This shit has to stop. If publish or perish is going to continue to be a thing, maybe just switch to publishing negative results. It's not sexy, but it's valuable too.

Yes, they'd get retracted if caught and be a black mark / destroy a career if someone did something like this and it was proven they intentionally faked it.

What often happens though is that the paper will get retracted but then the authors play hot potato in blaming different people for faking it and it turns into a whole mess where you don't always know (i.e. cannot prove) exactly who did the faking.

"Someone else faked it and I missed it, just like all the reviewers!!"

> If publish or perish is going to continue to be a thing, maybe just switch to publishing negative results. It's not sexy, but it's valuable too.

If we are to be allowed to publish papers from negative results then I should go and dig out my PhD notes and get writing!

Seriously, though, I started noticing the many broken incentives early on, after a explanation about "impact factor" from an older PhD student in our lab.

Same here - I went into science thinking that it would be all about trying new ideas and pushing the boundaries of what we know. And while in a certain sense that is still present, it takes a back seat to the perverse incentives you describe.

This was one of (and unfortunately not the only) reason I decided to leave academia after my Ph.D., leaving my field entirely and going into software engineering rather than taking a postdoc position. Academia isn't for everyone, and neither is industry... but wow, I sure am happier where I am now.

>This shit has to stop. If publish or perish is going to continue to be a thing, maybe just switch to publishing negative results. It's not sexy, but it's valuable too.

I had an argument recently about why fundamental research was important. The other person insisted that money should only be spent on endeavors where the outcome is known to benefit humanity. They couldn't accept, despite not having any science or math training, that sometimes we study things that start off seemingly useless and become important, or that we sometimes stumble upon things. (And this is why we should spend money on science in general).

Think about all the scientifically illiterate politicians who control money and funding who probably have similar views.

I can totally understand how we let things get this bad. If you don't provide a positive result, you might be facing unemployment. It's not about advancing our knowledge anymore.

Why spend money on the military if we’re not at war?

I published a number of crystal structures using a powerful X ray diffractor and high quality crystals where H density was both found and modeled. Modeling H density in and of itself is in no way a sign of fake data, it is just a rare occurrence.

Hang on, isn't a model by necessity not the same as the thing it's modeling? Obviously a photo is sort of a model, but depending on what the subject is, you can visually verify it. If it's a software model, and say you enter lots of parameters, I have to wonder how valid that model is.. How can it be verified?

The way you solve x-ray diffraction structures is that you refine your model until the electron density matches.

Thanks.. But until your electron density matches what?

You change the position and identity of the atoms in your model until there is very little residual electron density left remaining.

These are words, but I don't see how they prove the model. I guess it's like some sort of check after the operation is complete?

Why does a low residual electron density prove the model? What is electron density anyway? And how do we measure it?

Yes humans communicate using "words". Nothing is ever "proven" in science since there is always an arbitrarily complex hypothesis than can explain captured data. Instead, the weight of evidence is considered.

Anyway it seems like this whole conversation is out of your depth currently. There are many good X-ray crystallography texts available if your interest continues.

Sure. But in this case there was no claim of a sufficiently strong X-ray source for this, so it fell flat on its face.

The X-ray I used was "in house". The OP didn't mention what the X-ray source used was so the absence of mentioning "sufficiently strong" isn't proof.

"optimize a predicted structure of their molecule using density functional theory (DFT), then draw it like a crystal structure would be displayed (a 3D plot with error ellipsoids at each atom, with probable bonds shown)"

why not just publish this instead. surely an optimized model is still valuable.

Typically not on their own - it's very easy to calculate an optimized structure that is incorrect, so you typically want some sort of justification for your structure. Typically you would be calculating one of these geometries along the way to calculating other interesting properties to either compare with experiment or make new predictions (or both). The "gold standard" is... comparison with a crystal structure from X-ray diffraction.

In my field (small inorganic molecules used as catalysts), I typically calculated the structure along the way to calculating the molecular orbitals for describing the electronic structure (important for understanding many properties of the catalyst and its reactivity) and predicting the infrared and visible spectra (useful for comparison with experiment and for explaining the electronic and magnetic properties of the molecule). It's the whole picture that's valuable, not just the structure - and the more experimental properties you include, the harder it is to fake the whole thing.

I absolutely agree. Most of the time, getting very good DFT results is actually far more difficult that getting the experimental results of cystollography itself. They're both quite hard, and it depends on which techniques are applied, but getting computational methods to match with great quantum accuracy is very, *very* difficult. I know more about condensed matter physics and GW Bethe Salpeter approaches etc, than proteins, but the difficulty in getting good computational methods to work is still out of reach for ~95% of scientists even in that field.

Not to say this disproves anything, but you can definitely see hydrogen atoms in extremely high resolution crystal structures- typically with resolution below 1A, which is rare for proteins but should not be uncommon in inorganic crystals (right?)

ANy time you have a 3D structure (model) you can run the equations in forward mode to create the structure factors of the crystal that would have created that model.

Most of the time hydrogens are placed in calculated positions because you know where they are and it also reduces the number of parameters needing refinement. In good datasets hydrogens will show up in the density map (aren't all datasets good now in the age of area detectors), and you can refine (possibly with restraints) those hydrogens that you are uncertain about.

Raleigh's criterion says hydrogens should be just on the cusp, at least with home source xrays. Not sure about synchotron.

I mean, I know this is possible- I've definitely seen maps from very high quality (<.8A) crystals that clearly had defined bumps on the heavy atoms that were where you'd expect the hydrogens to be, and later on, neutron scattering confirmed it. But the only times I've seen this it was world-class competitive groups trying to elucidate the specific enzymatic pathway for a protein of industrial interest.

Yes, with a powerful enough source you can resolve protons, that's true. They didn't make such a claim - they listed a run-of-the-mill instrument in their methods section that would not have been capable of this.

> the author attempted to fake crystallography data

That is not what fake data is. It might be a misrepresentation is the author tries to pass it out as experimental results, but it is not fake, i.e. it came from a real study and actually has some meaning (though not much if it is indeed an unstable structure that can just be observed in a computer, and it is ground for rejection depending on the journal).

I mean, I don’t have much patience with people over-hyping their results, but fake data of the common sort is either photoshopped graphs, re-used spectra coming from other studies or compounds, or things like that. And this definitely happens. But the “it came from a simulation, therefore it’s fake” argument is very unhelpful in general.

A spectrum from a simulation is not fake, it’s just a simulation result.

It is fake if it's being misrepresented as experimental results instead of just a simulation

Again, that is not what this word means. It’s misrepresented data, not fake. And yes, the distinction matters.

This is also not what the article is about. The problem they face is completely fabricated data, not DFT structures in a poorly worded paper.

Of all places, this is probably the last one where I would expect this argument.

by this metric no data is fake. It's just misrepresented and was actually numbers I pulled out of my ass.

It is arguable that it is more than a misrepresentation if the author's intent is to fool the reveiwers. If so, it is "fraudulent data".

Intent to deceive is what makes it a fake/fraud. No?

Most of these are probably not even possible to catch, if the perpetrator has a basic understanding of the theory they could generate those correctly no problem.

Hence why fake it till you make it is so popular

You can see the hydrogens if you have enough resolution (below 1A). These days you start to see such good resolutions pretty often.

Here are two articles supporting my point as you don't seem to believe it. I also recommend you go look at PDB and select structures below 1A so you can see for yourself.

https://bmcresnotes.biomedcentral.com/articles/10.1186/1756-...

https://www.science.org/doi/10.1126/sciadv.1600192 """ Textbooks and teaching classes still emphasize that hydrogen atoms cannot be located with x-rays close to heavy elements; instead, neutron diffraction is needed. We show that, contrary to widespread expectation, hydrogen atoms can be located very accurately using x-ray diffraction, yielding bond lengths involving hydrogen atoms (A–H) that are in agreement with results from neutron diffraction mostly within a single standard deviation"""

Reply on news.ycombinator.com