Detecting model weight tampering in GPU memory (Rome attacks)

stealthium.io

Curious.

This feels like a tool to add to an LLM, being able to modify it's own weights, to be able to relearn, refine its own understanding.

Either that or force "standardisation" where one LLM (or actor) imposes its weights on others by rewriting them.

Not to jump on you, but it just irritates me when people anthropomorphize LLMs...

LLMs don't "understand".

Don't fall for the marketing and overstate their ability.

I don't mean to anthropomorphize. I apologize for my verbiage, language can be fuzzy.

How about 'functional control over their activated features', as opposed to 'understanding'.

It's a bit of a mouthful... :)

Doesn't this necessitate installing a malicious python module, which is pretty unlikely unless a supply chain attack occurs right?

Yes - the start of this attack would most likely be a supply chain attack. Good thing we haven't seen many PyPI attacks this year hey? Imagine if the LiteLLM attack had something like this inside of it versus the more noticeable credential stealer the bad versions then had

Reply on news.ycombinator.com