Author-paid publication fees corrupt science and should be abandoned

What if science and politics really are corrupt enough to corroborate the conspiracy theories? We need an International Journal of Conspiracy Studies right now!

Funny how the unabashedly partisan are the ones who make partisan statements in the same breath as their denouncements of partisanship.

Declining trust in science is not because of lack of open-access publications.

The right is not ignoring research because it is behind a paywall or because it hasn’t been reproduced. In fact, the right has chosen to willingly ignore research that has been painstakingly reproduced, all in service of the god of greed and ideology.

I'm not sure about the direction of causality here. Anti-vax people or climate change deniers also object to well-replicated results and would believe what they want to believe anyways.

I don't personally see a solution other than student loan money being much harder to get.

There is so much bad research because there are too many researchers.

For myself, I think I am smart enough to have worked on a PhD in economics but not smart enough to have done anything worthwhile or to have advanced the field. I am certain I would not have. Then in the face of this massive student loan debt I would have produced junk research and try to game the system to make the investment worthwhile. The incentives are all ridiculous in the current system for most.

The idea politicians will cut student loan money in order to produce better scientific research would be a good way to optimize for losing an election.

Personally, I don't think there is a way out. If anything it probably just gets worse.

Get paid to answer important questions instead of getting paid to provide sought-after results. I think that's the end-point, the "how we get there" is now the question!

Interested to hear what you mean when you say "turn clinical researchers into real scientists." Are you thinking of expanding the MSTP? Or just better statistical literacy for physicians? Or something else entirely?

I was told long ago that the first rule of evaluating a medical treatment is to ignore all the clinical studies because nothing useful comes out of them.

I don’t know if that’s actually the case but that seems to be the prevailing thought. Which is awful, because it should be the opposite. Clinical studies should be the gold standard. They’re often conducted by people who are better doctors than scientists though. A zero-blind trial of n=4 might perhaps not be the definitive study the doctor had hoped.

(Not to pick on doctors. My wife’s one. Software developers often fall into the same common “I’m good at this one hard in-demand skill so I must be good at all of them” trap.)

> So IMO, the first reform to conduct is to turn clinical researchers into real scientists. That won't be easy.

Mostly, there are not enough hours in a life to be both a good doctor and a good scientist. The few MD-PhDs who give equal weight to both areas are both brilliant and extremely driven. Multidisciplinary teams seem to have a better chance at success.

I guess if the scientific education was done early (undergraduate at the latest) dual training could work. Once medical training and practice starts there isn't a lot of time left over. And early / non-practicing science education, for whatever reason, doesn't seem to be very effective.

I am an MD/PhD, and I agree that a lot of the research done by clinicians is self-serving and useless. But most medical/health research is not done by physicians who are typically quite busy and generally not motivated to do anything that involves statistics and takes time and effort. Also, most clinician-led research is simply ignored because it tends to be overridden by the results of large clinical trials usually conducted by professional scientists. Where I see major problems are in two fields. The first is epidemiology (which receives about 30% of Research funding) where most of the research is done by non-clinicians, often involves data dredging and posthoc theorizing motivated by easy access to larger amount of pre-collected data and the availability of statistical software that permits fitting black box models that typically show whatever results are good for publication. This field is the source of contradictory results that we read every day in the news where something and its opposite are both good and bad for every kind of imaginable illness or symptom -. The other is a lot of basic sciences research which typically represent about 60% of all funded research. The great majority of that research is baseless as was shown by effort to replicate many of the findings. Billions of dollars are spent searching for genetic and other causes of illness using small poorly designed studies that can neither prove or disprove anything. So the problem is much bigger than just poorly trained and busy clinicians trying to prove a pet hypothesis. The problem is everywhere. Edited to fix typos

Uhh, the elephant in the room for CS/ML publications is that a lot of people hardcore lie on their results page since almost no one tries to reproduce anything.

I've witnessed folks straight out of their undergrad with literally 1 workshop paper at an ACL track conference get thrown 150K starting offers by no-name startups. Imagine how important it must be to get your first NeurIPS main conference publication. The ROI likely amounts to 300K+ per year, so if anything, I'm surprised that cheating/lies in the academy aren't even more brazen. Dynamics like this are why both Stanford and Harvard's presidents had to step-down for gross academic plagerism/misconduct.

1. https://www.science.org/doi/10.1126/science.359.6377.725 (2018)

2. https://www.technologyreview.com/2020/11/12/1011944/artifici... (2020)

3. https://www.nature.com/articles/d41586-023-03817-6 (2023, Nature)

Double blind reviewing is not the norm in computer science. You will sometimes see it, but it depends on the subfield, the specific conference/journal, and even the year. In my experience as an author/reviewer, the submissions have been anonymous maybe 10% of the time.

And the biggest issue I see in CS is the focus on gatekeeping. When many (most?) papers are published in conferences with a single round of reviews, peer reviewing becomes more about accepting/rejecting the paper than improving it. The average review I receive in bioinformatics is more useful to me than the average review in CS.

> If you're hiring in your own field (which you should), you don't have to read the papers. You're already familiar with most of them, and judging by the venue their published in, you also know their "level", so to speak.

You're judging by reputation rather than substance. That opens up the possibility of manipulating reputation to achieve career goals instead of doing the hard work.

That you're struck that this is someone who "doesn't hire faculty members" is exactly the best thing.

The academy is so corrupt it should be "shattered into a million pieces and scattered into the wind". H Index ego, P hacking, lazy lies because "no one tries to reproduce", citation cartels, institutionalized systemic academic dishonesty/plagiarism at the highest levels (presidents of top universities and likely their underlings), "reviewer #2", etc. It's all bunk, and most of it is designed to take bright-eyed Ph.D's and grind their soul to bits before they can realize that they can both out-earn and out-produce (in an objective sense) most of their professors by simply leaving the academy.

Being ineligible for tenure should be a badge of honor, as the whole system should be abolished and the world would be far better for it - particularly, science would advance far faster.

Far more of HN needs to read Paul Feyerabend and actually internalize what "epistemological anarchism" means. The academy does not hold any particularly valid claim to being closer to observing "objective truth" than anyone else. The allegory of the cave/divided line (the primary justifications for a lot of academic wankery) was one of the original lies made by a nascent and authoritarian philosopher class. We should have abandoned these ideas long ago.

Oh, and if you leave the STEM fields, the academy goes into full on "charlatan" mode. I go into most social sciences and find a wasteland of footnotes on Foucault and Lacan. That Situationalist International was so successful that the academy is still choked by an obsession with french fashionable nonsense is an indictment of the whole of the academy in the west. It's all intellectually bankrupt.

You sound like review 2: speaks from authority and gives criticism with little substance.

  > You're already familiar with most of them

In what way? Their prestige? If I went about outsourcing my opinions of people to others I'd hold many criminals and objectional people in high regard while also looking down at many great men. Even if this works 80% of the time you trust *but verify*. Have some actual confidence in yourself. Stop outsourcing your intellectualism.

  > I've served on several program committees

Then maybe stop complaining and start critiquing.

You speak with an air of authority and that we should trust you but give no actual reason why. Use your experience and expertise to educate us. Don't just say I'm wrong, say why I'm wrong. Convince me you're right not through ethos, but logos. This is what everyone hates about review 2. Don't become him.

I believe you've also missed the substance of what I originally suggested. You're so caught up in the standard way you injected it into my comment without even noticing. You forgot the point all of this. It to climb the ladder of academia, get grants, and all that. It is about advancing science and moving humanity forward. The way to measure this isn't meaningful at our disposal. You can't measure this by citations or indices, by publications or venues. Again, they can be a signal (I'm not saying to throw out the baby with the bath water), but I don't believe you're naive enough to not be aware of how frequent progress is made by those who fail on those measures, how frequent it comes from a dark horse. This should be enough to suggest that there's more to it than what we usually consider. Academia has always been about challenging the status quo, even if it is just to verify. The problem with academia is it's created to give refuge to those who want to spend their lives dreaming but has become so structured you can only dream a certain way. I get why. It's hard to differentiate useless pipe dreams from those that push us forward. But in creating the structure to prevent people from taking a free ride you created the monster you tried to stop. Only now it has more power.

So don't just wave your hands dismissing, elaborate. Be academic not arrogant.

> I'm happy with blind reviews.

Of course you're happy as the reviewer. I would be too if I could criticize completely anonymously and have it counted. Me as the reviewee? Not so much.

I always thought it was funny: in the US, it's a fundamental right that you should not be criticized anonymously (6th amendment). Ostensibly this is to prevent laundered dictatorial rule. And in academia it's the norm lol. Makes you wonder doesn't it.

I too have served on faculty search committees and reviewed scores of papers. Many members of the committees only counted publications. I read some papers and commented on them without much effect.

It is in nobody's individual interest to switch from an established journal to a new one before at least half of the rest have switched, so nobody does it. Scientists are also being squeezed within an inch of their lives by the career advancement / funding system leaving no time for the organization of this kind of change.

You are clearly unfamiliar with biomed research. Journals do have prestige by themselves, by now. And it's not so much reviewers but rather the acceptance rate on cursory read by an (sometimes anonymous) editor that maintains it.

The problem with gold OA is the proliferation of low-quality spam, and the recommended remedy is to both restore exclusivity (raise the barrier to entry) without supporting excessive (monopolistic, exploitative) journal subscription fees. The article's recommendation to build up society journals has a decent chance of accomplishing this. It won't prevent spam from getting published, but the only essential aspect is to keep the spam easily identifiable (e.g., published by MDPI) so it can be safely ignored. The www seems to facilitate the spam business model so I don't think the spam will go away entirely.

The scientific community, or the ministry of science of some country, or a university - might find that paying for peer review, for example, might be more effective in promoting good science than paying for publication.

In what I was imagining, the income was from people paying to be considered for inclusion on the list. Reading the list is free.

This is great! I'm trying to think of any disadvantage.

I lost a lot of respect for science when my advisor explained that it didn't matter that the cutting edge work I wanted to extend was being done by Some Guy On A Forum, I wasn't allowed to to cite them, and a pity too, because it was probably correct.

> Corporations like Elsevier have successfully executed takeovers of research centers (universities) and made journal submission a mandatory rite of passage for academics joining the state-mandated academia funnel.

I...think the causality on this is wrong.

"Publish or perish" long predates Elsevier's rapacious consumption of journals. The reason they did that in the first place is because they already knew it was, in effect, a captive audience.

Filtering should happen after publication, and be built into platforms like ArXiv.

> It's a filter thing.

I'm reminded of this quote from Ben Thompson:

In short, the analog world was defined by scarcity, which meant distribution of scarce goods was the locus of power; the digital world is defined by abundance, which means discovery of what you actually want to see is the locus of power. The result is that consumers have access to anything, which is to say that nothing is special; everything has been flattened.

Instead of filter, I tend to say curate. There are so many papers being created no institution has the time to vet and curate. They want to use open access repositories and journals, but they don't want to bring hundreds of thousands unvetted papers and books into their local catalogs.

There's also the prestige of certain journals, but like you touched on, at a meta level that's just curation. Citations and other scorings are attempts to address this issue, but those end up heavily weighted towards older publications that it's hard for new, possibly better research to bubble up.

> take away the government grants and people lose the incentive to engage in that kind of scientific fraud

And the incentives and resources to engage in scientific activities in case of other fields.

On the other hand, the ML community is especially big on papers with cherry picked results and things that can't be replicated.

Our group always pre-publish to arXiv, as a way of saying "we did this first/already" because we usually suspect, and half the times are correct, that our competitor are on a similar research. But as soon as we arXiv, we start the hunt to publish with as high Impact Factor as we can. Why? Almost 100% grants applications are going to evaluate you for the weight of your publications, and some of them doesn't even consider indexed publications below Quartile 1 or 2. The IF of the journal is a direct proxy for your work quality.

In our experience, arXiv doesn't peer review at all. You get some twitter posts, messages of congratulations, at most a couple of questions. But nothing like the average peer seriously reviewing your paper.

I wouldn't pay a dime to publish if my salary wasn't affected (e.g. being Linus). I would submit to arXiv, Github, Twitter or even a "Tell HN", and probably reach a larger audience. In fact, the journals ask you to promote your papers on your social networks, because they know theirs doesn't matter unless you are Science or Nature. But sadly that's not how the scientific world works.

The problem is that the rest of the world does not really like reviewing stuff. A few people hint frauds, which is a great service, but that’s about it.

> And how do you even guarantee this, 'accurate' and 'unbiased'? Any form of peer review will always have a slant, no matter how 'objective' it is intended to be.

Peer review of peer review. Instead of a couple of scientists looking at it, the peer-review process should be entirely open. So other people can scrutinize it.

> How do you prevent movement to an equilibrium where reviewers give favorable reviews to the reviewers of their own papers?

Peer-review of peer review. If all peer review is public, then people can analyse the trends.

Well, if the paper used falsified data, the authors should be punished. We can't expect all falsified data to be found, as some people are really good at falsification.

> There's no a priori reason why the expected success rate of research projects should be 50% and not, say, 1% or 99%.

Humans are really bad at predicting the future, so I'd argue that the large majority should be between 49% and 51%. A 99% success rate would imply that humans can predict the future, which I think is pretty much the opposite of how science should work.

You might want to get a paper bag to scream into: https://arxiv.org/abs/2408.06292

Or organize a fraudulent citation ring.