Edit: from all this talk of reproducibility, I wonder what percentage of cutting edge ML research is reproducible (either from lack of public training sets / not enough compute)
Edit: from all this talk of reproducibility, I wonder what percentage of cutting edge ML research is reproducible (either from lack of public training sets / not enough compute)
Other CS subfields that get a lot of criticism are "network science" and bioinformatics.
That generally isn't enough to get published nowadays though, at least in a simple sense (in a broader sense that process might describe all research, of course). To get published requires some deeper demonstration of a new kind of method that not only works, but is superior to all that preceded it in some important way. In other words you show your new method compared to other methods, where yours must be better. Obviously, bad research here can show a supposed advantage by either doing a unfair job applying competing methods, or overfitting their new methods. Or both. As I understand it, the first one is quite common: comparing a poorly-tuned old method to a carefully-tuned new method.
Clinical trials can often be flawed, even if the stats are fine, just in how they sample. For example, women are often excluded from trials due to hormonal changes, but how drugs impact women is really important! Participants are also typically drawn from specific locations, and so may not be representative of people with different diets, lifestyles, and environmental factors.
Physics has its own controversies, though not always directly related to replication. For example, Harry Collins recounts the social factors involved in the discovery of gravitational waves: https://blogs.sciencemag.org/books/2017/03/28/harry-collins-...