Hacker News Reader

It’s stealing the last layer (softmax head), not an arbitrary part, also it targets “production models whose APIs expose full logprobs, or a logit bias”. Not all language model APIs have these features and this characterizes what APIs can be targeted and what can’t. These important pieces of information should have been written in the title or abstract rather than “typical API access”.

View on HN

It's still significant. When the Softmax head is the transpose of the embedding matrix, the proposed method enables extraction of the entire matrix of pretrained token embeddings from a black-box model at a shockingly low cost. If I understood this right, there's a lot of valuable information in those embeddings!

That's not always true, many models have separate weights for the embeddings and classifier heads

You're right. I edited my comment to qualify it with "when". Thank you!

Reply on news.ycombinator.com