Hacker News Reader

Comment by LudwigNagasena | Hacker News Reader

That’s a weird change. Publishing reviews alongside papers is good. Removing the “stamp of approval” from the whole process is weird. Imagine if every student who managed to enroll into a university would receive a diploma; and every course would only provide a review of a student instead of grades. That would just make everything much harder.

Yes, it may seem wrong to put things into black and white buckets; but many our decisions inevitably are binary and we have to make them no matter how hard it is.

View on HN

The presence in a journal is not a grade. It is a "pass".

Perhaps some journals only let grade "A" papers pass, but sometimes that same journal has an editor who might let their buddy's article slip through. If one is part of the old boys club, this is a nice situation.

What eLife wants are signed grades from editors and reviewers. The editor is supposed to attach a brief summary in a few sentences summarizing the reviews. Hopefully one might bother to read a sentence or two to evaluate a paper instead of merely looking at the cover of the journal.

eLife's move here is basically a statement that the system is corrupt. Those wanting to fight the corruption are trying to increase transparency and reduce arbitrary decisions.

The whole point is that journal prestige is harmful and fake. It's prestige by association. That's not how science should work.

The problem is that science has not figured out how to work without this system.

Sure it has. The modern “scientific publishing” industry is much younger than science.

Being able to produce a bazzilion papers of low quality is also new.

Yeah, it's the formal academic hiring system that hasn't yet figured out how to work without this. But of course, that's what eLife is trying to figure out: they're trying to separate recognition (in the published assessment) from publication. The big question will be whether those assessments will be able to play that role in academic rewards, because that's the root of the incentive system that preserves the status quo.

Lolol

Yeah we have. It's called reading the paper and applying judgement.

In some fields, there are thousands of submissions daily to Arxiv. Good luck staying up to date.

That's not a problem to be fixed through fake journal prestige.

Maybe you do that.

But my HR department which made me write "publish X papers in top-tier journals (see list)" as a condition in my probation agreement most certainly does not.

And alas, as long as I'm under probation, their opinion carries far more weight in the debate than any moral argument you could come up with.

Yes I do that, thank you.

Thanks for clearly stating how journal prestige is harmful to academics and scientists.

I didn't say I disagreed with you, let alone I approve of this situation.

The point was that utopian idealism that ignores reality on the ground is not as useful as you might think.

Your comment was as useful as me saying if I was president I'd give people free money.

When was the last time you were hired like that?

About 8 months ago :)

> It's called reading the paper and applying judgement.

Many important papers these days are interdisciplinary. Like, medical students doing bioinformatics and data analysis. But it’s impossible to be proficient in all the methods. So, many parts of these papers are impossible to assess for a wide audience, trust and faith in a journal quality becomes most important.

I don't do my science by faith.

Then you probably don’t do science in 2023.

The change just means that the stamp of approval is not the journal name, which is fine. If you think about it, that system is a bizarre and lazy way to attribute value to a piece of work. The book publishing model figures out which books are great without caring at all about who publishes something (for the most part)

They're not removing the stamp of approval:

> with a short editorial assessment of the work’s significance and rigour

However, not getting the stamp of approval will no longer lead to the work not being accessible to people.

But every paper can already be accessed by anyone if its authors decide to put it on arxiv, GitHub, their personal website or anywhere else they wish.

So does that mean that the only practical difference is that now when you list your papers for a promotion or tenure review you will write “eLlfe (approved)” instead of “eLife”? Then what is all the fuss about?

1. There’s still some hurdle before new-eLife will send your paper out for review. This is not necessarily “quality” or “impact” level; they say it will depend on whether they can provide a helpful and effective review.

People had very mixed experiences with old-eLife’s editorial decisions, and some felt it was a bit clubby: papers from some labs sailed through to acceptance while similar work from others got editorially rejected (without review) for seemingly minor reasons. Their policy against requesting new experiments could certainly be used as a cudgel: an editor can just say you’re not convinced by a control and the paper’s DOA.

Thus, they are both uniquely positioned to make sweeping changes and have people doubt them.

2. The reviews use a controlled vocabulary, so I think you’d say eLife (excellent approach, major impact) or whatever the words are.

Yes, but now the authors don't need to manually decide to also share their research elsewhere - something they've (in many, but not all, disciplines) been trained not to do as publishers traditionally wouldn't allow it.

The fuss is about publication and approval being seen as linked in the academic hivemind. Even if you're able to see them as separate steps yourself, that doesn't mean that the academic world at large sees them as such, so if this fails to gain widespread acceptance, the reputation of eLife-the-journal might suffer, and having been published there will no longer look as good on your CV as it currently does.

In other words, the people who are afraid of reputational loss are the ones who have benefited the most from the broken system of academic publishing.

I won't cry for them.

The main question is: will those people be able to prevent this change from happening. From the article, it looks like they've been able to delay it already.

It will also change the process from the perspective of an author. If you are not fully sure about the impact of your publication, you might start submitting it to the journal with the highest possible impact. And, after a rejection, continue submitting to the next best journal, until acceptance. The new process by eLife would hence massively cut the workload on authors, besides editors and reviewers.

> every course would only provide a review of a student instead of grades

Would it? You'd think that a hiring manager would rather see a description of what the student accomplished, and with what level of skill, rather than a rather opaque letter grade.

Of course, that would make it tough to run "cattle car" courses with thousands of students being graded with multiple-choice tests. I don't think that would be a bad thing, either.

> Of course, that would make it tough to run "cattle car" courses with thousands of students being graded with multiple-choice tests.

It will be easier with GPT providing the evaluation.

Ugh stop. Not every conversation has to be about GPT.

What makes you think it actually can provide an evaluation?

Reply on news.ycombinator.com