Also, if you're decoding TLS on the enclave, wouldn't that imply that you're parsing HTTP and JSON on the GPU itself? Very interesting if true.
Also, if you're decoding TLS on the enclave, wouldn't that imply that you're parsing HTTP and JSON on the GPU itself? Very interesting if true.
HTTP parsing and application logic happens on the CPU like normal. The GPU runs CUDA just like any other app, after it's integrity is verified by the CPU. Data on the PCIe bus is encrypted between the CPU and GPU too.
[1] https://github.com/NVIDIA/nvtrust/blob/main/guest_tools/atte...
Does the CPU have the ability to see unencrypted data?
When the enclave starts, the CPU does a few things:
1. The CPU does a key exchange with the GPU (in confidential compute mode [1]) to derive a key to encrypt data over PCIe
2. The CPU verifies the integrity of the GPU against NVIDIA's root of trust [2]
[1] https://developer.nvidia.com/blog/confidential-computing-on-...
[2] https://github.com/tinfoilsh/cvmimage/blob/b65ced8796e8a8687...
edit: formatting
Yes.
>How do you load balance or front end all of this effectively?
We don't, atleast not yet. That's why all our model endpoints have different subdomains. In the next couple months, we're planning to generate a keypair inside the enclave using HPKE that will be used to encrypt the data, as I described in this comment: https://news.ycombinator.com/item?id=43996849