Hacker News Reader

Comment by htrp | Hacker News Reader

> Run:ai has developed an orchestration and virtualization software layer tailored to the unique needs of AI workloads running on GPUs and similar chipsets. Run:ai’s Kubernetes-based container platform for AI clouds efficiently pools and shares GPUs by automatically assigning the necessary amount of computing power – from fractions of GPUs, to multiple GPUs, to multiple nodes of GPUs.

Basically Nvidia coming out with managed AI services on DGX cloud

View on HN

Apparently the DGX cloud is hosted first on Oracle, moving to Azure and hosting on Google soon...

From 2023: https://nvidianews.nvidia.com/news/nvidia-launches-dgx-cloud...

"NVIDIA is partnering with leading cloud service providers to host DGX Cloud infrastructure, starting with Oracle Cloud Infrastructure (OCI). Its OCI Supercluster provides a purpose-built RDMA network, bare-metal compute and high-performance local and block storage that can scale to superclusters of over 32,000 GPUs.

Microsoft Azure is expected to begin hosting DGX Cloud next quarter, and the service will soon expand to Google Cloud and more."

There are also self-hosted DGX Grace Blackwell systems releasing later this year / next year.

DGX cloud is nvidia's cloud offering (you can think of it like a snowflake/databricks in that it runs on top of other clouds), with allegedly the reference design architectures vs someone just slapping some H100s in a server rack.

Run ai is not a mature product at this point though.

Nvidia definitely overpaid an order of mangitude here, this setup's a month ish work for any semi-skilled SWE who is familiar with RDMA, SR-IOV etc.

Reply on news.ycombinator.com