Is this only an issue if you are using agents? Or are there any theoretical risks of serving a model over say llama server (llama.cpp) and accessing it for chat in browser on machines on your local network?
If we're playing the risk game and we assume that this capability exists and is exploitable, then yeah your chats would be just as exposed to the capability as agentic output. So you might be told that the sky is red instead of blue, but only if you previously asked about the freezing point of water.