Is the ChatGPT API Refusing to Summarize Academic Papers?

mattmazur.com

11 points·saeedesmaili··7 comments

Q: Is chatgpt refusing to summarize academic papers?

A: No. He was using a weird prompt which was sometimes causing problems.

Panic over.

Absolutely. If anything, ChatGPT is saying it can't "take a deep breath". What a dumb prompt that was.

It's from this paper (albeit not applied to ChatGPT?): https://arxiv.org/abs/2309.03409

My gut tells me it's the last line about writing a "blog post" which could be interpreted as a guardrail-violating imperative.

That line also begins with "Now, take a deep breath" which an LLM cannot do.

It likely could be, not to mention the prompt includes "Do not mention the authors names" which probably makes it thinks you are trying to plagiarize

I'm a little surprised by the lack of the solution or deeper analysis. It seems like there are some hints towards figuring out the problem.

To more closely confirm the actual reason, the easiest bet would be to prompt further on rejection, or adding to the prompt "If you can't do something, please indicate why you can't"

I am almost certain that it's because of the wording of the prompt. I don't think it's that the prompt is "too complex." Instead, the prompt itself is self-contradictory. It tells the model it is to summarize, but at the end asks for it to write a blog post. It gets confused and just says that it can't do that because it is acting as a "summarizer" and not a "blog writer"

Reply on news.ycombinator.com