How do you deal with data persistence for storing the documents/vectorDB inside a Nitro Enclave? I would assume that you as the SaaS vendor are unable to decrypt the sensitive documents inside the enclave or see a users chat history?
When the search endpoint is called the encrypted embeddings are pulled into the NE and decrypted. They are then loaded into a vector db in-memory and the search is executed all within the NE. This adds some latency but it’s more secure because embeddings are only accessible to the enclave.
In the case of chat history it is never stored by the API. The developer can develop their own client side. With CapeChat we keep chat history local on the device.