I have the same plans for ctranslate2[0] and Willow Inference Server[1]!

[0] - https://github.com/OpenNMT/CTranslate2

[1] - https://heywillow.io/components/willow-inference-server/