Transparent GPU Sharing in Container Clouds for Deep Learning Workloads
usenix.org
usenix.org
https://arxiv.org/pdf/2203.09040.pdf
"As one of the most competitive kernel auto-tuning framework, TVM comes with a built-in assumption of single-tenant execution setting, the tuning configuration of which aims to saturate all SMs and memory bandwidth of the GPU. Such assumption and the single-tenant targeted tuning however becomes unsuitable for multi-kernel concurrent execution with partial resource available for each kernel. According to [7], the maximum throughput gap comparing single-tenant vs. multi-tenant tuned configurations for the same computing kernel could reach 5× difference."
TIL: somehow I had assumed Apache TVM permitted multi-tenancy.