Comment by stasomatic | Hacker News Reader

Comment by stasomatic | Hacker News Reader

Parent Full thread

That's two AI smells. AI likes to say vacuous punchy statements like "the final takeaway" or "here's the rub". Then contrastive parallelism "connections, not CPU". Contrastive parallelism should scarcely exist in technical writing, regardless of whether it's AI generated.

Aren't the LLMs trained on a massive corpus of human written texts? If that stands, then they are doing what they were asked, kind of? I am also not a fan of the fluff words that Claude pollutes the context with, it's a bit much, I could live 30% less but it doesn't want to read its instructions in the .md

If the author wanted, they could ask their "AI" to write the article as a caveman, no? I, simply for fun, slop coded up a skill so that I can ask Claude to reply as Samual L. Jackson, or Kramer, etc.

I also wonder if there is an "AI language barrier", likely not in this particular article, but let's say there was a published article by a researcher whose native language isn't English. What happens with the translation? I realize I am asking naive things :)

There are distinctive patterns of language use that can be powerful when used sparingly. LLMs trained on a massive copus of human communication, picked effective patterns, and overuses those patterns to the point that it feels both artificial and underwhelming.

I wonder if the human training process amplified this effect. I like to imagine there's an OpenAI RLHF trainer in Kenya whose name we'll never know who really likes this style of writing and is solely responsible for LLM voice. I know that's not how it works, but it would make for a fun story.

And you know what? That's not annoying--that's brave.

There's a post training.step where they get humans to interact with it.

As I understand it, there is (or was) a step where they ask people what's the 'better' response.

These linguistic forms sound good the first time you hear themz even if they are rare in real speech, so rapidly got trained in.

Now they distill off previous models, I imagine these weird linguistic forms are quite hard to get rid of.

If there are human knob turners, they can’t not be lurking here. If they read the room, dial down the knobs.

>Aren't the LLMs trained on a massive corpus of human written texts? If that stands, then they are doing what they were asked, kind of?

I think you might be interested in reading in training data generation, training, and post training papers/articles. I think you might be surprised at how much intervention there is on some of these levels.

Reply on news.ycombinator.com