Compliance is pretty obvious and even if you don't store the feature dictionaries and only the vectors, that's a hard conversation to have with the compliance team. I GET IT, without the feature dictionaries the vectors are useless, I KNOW this is how it works from a technical point of view, but the compliance team still won't sign off on it. That's just the way of the world.
And much much more important is performance, uncompressed high dimensional features are huge and even if you use run-length-encoding or sparse vector storage in the protocol plus some lossless compression, I have trouble keeping the GPU fed from disk, let alone over the network, it's going to be multiple orders of magnitude too slow. If the benefits is not claimed to be streaming but are fast vector similarity, keep in mind I can do cosine similarity on literally millions of vectors a second on a single CPU core using vanilla numpy, this was fast enough for me to implement realtime face recognition vector search for Dubai airport, so pretty high scale operations.
I've love a self hosted version of this, optimised for I/O throughput to the GPU. That would be great.