Hacker News Reader

Comment by Solvency | Hacker News Reader

Why has the term grokking been formalized like this? Honest question. Why can't we stick with normal words like "learning" or "understanding"? It just seems like a slow creep of needless semantic complexity in an already esoteric and niche space.

View on HN

The term "grokking" has been used for something like "deep understanding" for many decades now, after it's introduction in an early 1960's novel. Perhaps not falling into completely general use, but popular within a large swath of technical communities, certainly.

I'm not sure it's particularly well targeted here, but they are going for something deeper than "learning" which already has a standard usage in the field.

But this paper isn't using "Grok" in that standard slang fashion but in a very specific fashion, which I think the gp can reasonably complain about.

How would using common words suggested by ggp avoid this issue? It's a usual occurrence for terms of art to be repurposed words having somewhat related usages elsewhere. It all sounds like an empty complaint.

Grokking in ML has been around for quite a while. First heard the term being used at a conference in 2017.

The term classifies a different type of characteristic of the learning trajectory, so just using 'learning' would be missing that distinction. There needs to be a separate term for 'long stretch of epochs for which there is a large, almost perfect, separation between train and test accuracy' ... and I would hate to write that over and over again, so there's a single term for it. In some respects, it's maybe problematic for causing a namespace conflict for grokking in this sense, vs grokking in the more 'normal, human' sense.

Because if you've spent 80 hours a week researching, you want some fun in the 85th hour of your work week spent writing the research up.

A stronger hypothesis: the main goal of most AI researchers is to make other AI researchers think they are cool, and using a word from classic science fiction helps do that.

That seems obviously false to me, according to what I know about people in general, and researchers in general.

Grokking comes from Heinlein, and it is more than a synonym for both of those words.

Good intro here: https://en.m.wikipedia.org/wiki/Grok

Calling these spikes "epiphanies" might be a better term. I think grok is nice b/c it's only 4 letters.

I'm going to guess that in AI those terms have been so cheapened by now that they are useless.

because, in highly technical research area, there are zillons of somewhat similar effects. Some are interesting, some are not.

They need a word for each interesting effect. Sometimes they make up new words, sometime they reuse a rarely used word

It's also problematic because it implies a subjective "feeling" that we associate with sentience (a property of AGI) - not AI systems.

Yeah, this gratuitous use of fancy words induces cognitive load for me.

Or 'comprehension'.

Reply on news.ycombinator.com