354 karma · joined June 30, 2016
that's a little harsh :D
I don't think this is possible when you can't pool memory on the 40 series retail GPUs.
I'm afraid that unraid will run into similar problems eventually.
I wonder how the author feels about audits? At the end of the day, even if mullvad is logging, I trust them with my data far more than xfinity/comcast.
[1] https://mullvad.net/en/blog/2021/1/20/no-pii-or-privacy-leak...
I don't really understand the technical solution behind this statement. Isn't this just e2e encryption up until you decrypt the query to pass to the LLM? ML operating via homomorphic encryption is very far away and OpenChatKit is just a standard self-hosted LLM. This seems more like "self host on semi trusted azure compute that isn't owned by openai and the model performance will be far worse than gpt4"?
We understand that not everyone has time to contribute to public discourse, so #2 is optional.