Some scientists can't stop using AI to write research papers | Hacker News Reader

Some scientists can't stop using AI to write research papers

theregister.com

12 pointspseudolus11 comments

I've heard quite a few stories of non-English-fluent scientists using AI to write research papers - based on the belief that their poor English skill would lead to near-certain rejection of any paper they wrote. Vs. even a pretty-crappy AI can easily seem fluent.

So, question: Let's say it was 1949, poor-English-skills Prof. Grzegorz Kołodziejczyk was teaching mathematics at MIT, and he tasked Mrs. Jane Smith - of the Math Dept's clerical staff - with typing up his research paper on Szemerédi's theorem. "And might I ask you to correct and improve my poor English as you do that?"

Ethically, how would that differ from using AI in 2024?

Or, what if Prof. Kołodziejczyk wrote the research paper in his native Polish, hired a professional translator (specializing in mathematics, obviously), then only published the English version of it?

An important difference is the specific way that AI will fill in the blanks. A human translator can ask for clarification, in the best case understand the source material enough to figure out translation hitches, and in the worst case be held to blame. I think it would also be reasonable to credit the translator, whether or not that actually happens universally. An LLM doesn't "understand" anything, and just makes shit up. They make shit up so readily that they can't be trusted to ever not be doing that. Their use is also not being disclosed.

My experience is exactly the opposite. AI has way more domain-specific knowledge than any translator I could imagine.

I recently published effort [1] - it was on front of HN recently - and literally everything on that page was rewritten by chatgpt.

The flow is that I write it however I can, sometimes so badly that even a human professional would have difficulty to understand, then I ask chatgpt to go paragraph by paragraph and rewrite and smoothen out the text.

You can see how the website was reworked here:

https://chat.openai.com/share/e/10d7ba3f-f7eb-48cd-9250-d864...

GPT has way more domain knowledge in my area than most engineer friends I know, not to mention translators, and it managed to not just help with grammar, but with overall explanations I wrote.

Of course it fails at higher level concepts, and a plenty of things, but still - I can get more quality output from him in an hour than by working a week with a dedicated translator/editor.

[1] - http://kolinko.github.io/effort/

Sound like "use AI" is morally equivalent to "use a not-so-good secretary or translator". (I don't know the 1949 standards for crediting translators of research papers - but I'd bet that 99% of the blame for poor translation, by someone he'd specifically hired for the task, would fall on the Professor.)

Rather than all the hype and panic about "AI-written Research Papers!", it seems far more reasonable to say "We'll treat you having an AI write your paper the same as you having your kid sister write your paper - responsibility and blame are 100% on you. Maybe with some extra helpings of humiliation and contempt, if it turns out that your kid sister wasn't qualified, and you failed to realize that and fix everything."

But, back in the real world - hype and panic generate loads of lucrative clicks. So I'm not expecting intelligent policies any time soon.

It is a question of degree.

If one was to give the tool a complete written document and ask it to specifically correct spelling and grammatical mistakes along with identifying any sentences which are unclear... and leave the agency of fixing the issues on the researcher? No problem at all.

If one was to give the tool a complete written document and ask it to fix everything and then publish that, then you get into problems where there has been no check to make sure that the AI (or clerical staff) has not made any mistakes.

If one was to give an outline to the AI and some results and ask it to write the paper that has these findings... then there's a problem.

I would personally encourage people to do the first - even if a native speaker (I do it with some things that I've written).

Beyond that - having an AI generate content rather than identify issues or not take back the agency of fixing the issues back from the AI, that is where problems lie.

---

For some of my previous thoughts on this in the context of using a LLM for assistance in writing papers for school - https://news.ycombinator.com/item?id=39110737 and https://chat.openai.com/share/d891f9ac-923b-47a8-8de9-ab7301...

This seems like a strawman. I am not aware of any serious LLM critics who object to using transformers for machine translation or cleaning up English prose. LLMs are quite reliable at purely technical translation, and decent at editing provided they don't have to summarize anything.

What people object to is using LLMs to generate grist for academic paper mills, or alternatively to outsource peer review. The quality of academic papers is already pretty deplorable! I am quite confident LLMs (and excessive reliance on preprint servers) are making things much much worse.

Notice though, that the main post here suggests that "scientists can't stop using AI", shaming a bit scientists for using it, and not distinguishing if AI was used for redaction or for writing the whole article.

Exactly this. In my case, I can communicate in english quite fluently, but when it comes to explaining technical things in a form of a paper or any other specific form - my language skills are still lacking. GPT is a saviour here.

It will miss some things that a human technical reviewer would find of course, but for a first and second iteration on a text it's all I need.

I think in reality, Jane Smith would be a grad student who has a deep enough understanding of math to write a lucid essay.

The problem with a non-math clerical person or AI writing about a technical topic is the same problem clickbait science journalism has. They wildly misunderstand things, and sometimes even a single poor choice of words leads to completely false statements.

If there's a real truth being communicated, especially a new frontier paper not in any training set, then A.I. is going to just deteriorate the accuracy by fluffing it up with hallucinated mumbo jumbo.

I use A.I. in my writing all the time, but I do it the right way. I make a list of the real truths and arguments I want to make, I read the A.I.s essay, I grab the snippets I like, then I dictate vocally an essay on the fly, while skimming the snippets for the talking points. Then maybe I do that loop again to refine. A.I. is a chatty, weird talking coworker on drugs who you bounce ideas off, it's not a genius you can directly plagiarise from.

> The problem with a non-math clerical person or AI...

Try talking to an old professor or two, about just how capable any decent department's veteran clerical staff actually were, back in the day. They'd been immersed in the subject, daily, for years. And it wasn't an era of good opportunities for educated women, for the best & brightest to quickly move on to higher-status jobs.

In our future, someone is going to do a study that attempts to measure what I'm going to call the "AI shame index." This will be a measure of the delta between a person's stated usage of AI and the person's actual usage of AI.

Basically I think that there are all kinds of use cases for AI that people are discovering but are unwilling to say out loud because of a perceived cloud of problems associated with AI (e.g., hallucinations, privacy, etc.).

As use of AI becomes normalized, this index will shrink. But for now, I suspect that there quite a lot of closeted AI users. And I doubt that it's just limited to non-native speakers.

Reply on news.ycombinator.com