Hacker News Reader

Comment by fzxu22 | Hacker News Reader

Working on this: https://github.com/KevinXuxuxu/anon_proxy, a sort of anonymization proxy to use with LLM providers. It does model (OpenAI privacy filter) + regex PII detection, and replaces them back-and-forth for API requests and responses. With locally hosted detection model, no PII leaves your local environment. I find it very useful especially when you're working on sensitive documents (legal, tax, immigration etc.), hope you find it helpful as well :)

View on HN

This is very cool because it allows you to use any model. Obviously, it still lets the model and its operator see the entire context of the conversation.

I quite like Moxie's Confer[1] approach to just encrypt the whole thing in such a way that no one except the end-user sees the plaintext.

[1] https://confer.to/

That looks interesting. I would like to see them update the Privacy Policy and Terms to acknowledge that their service also works with an Apple ID or with another email. At present, it suggests that the only authentication allowed on your end is through Google's GMail.

It's a nice approach – if only Intel SGX were more trustworthy.

Thanks for the comment! And yes, redaction based measures will always face the trade-off between privacy vs intellegence you get out of LLM. e.g. provider will inevitably know you're in some sorts of legal/tax issue even without any PII. And for some case the intellegence you want will depend on LLM knowing some detail (e.g. your AGI when doing tax preparation).

On the other hand Moxie's Confer is really interesting! On first glance I thought it's using homomorphic encryption but it turns out to be based on hardware isolation. TIL +1

How does it handle “unredaction” in responses? E.g. let’s say the LLM does something with the document. You redacted its input, so it emits redacted content. Now what?

The way I handled it is by assigning the redacted tag an id which gets translated back to the saved PII in the output.

The proxy keeps 2-way mapping of identified PII and the redaction e.g. Jane Doe <-> <PERSON_1> so the process is reversable i.e. redactions from LLM response will be replaced back to the original, and it should feel transparent on user end. I'll add more detailed example in README to make it clear.

Reply on news.ycombinator.com