Pardon me not understanding what this is about?
Basically SETI for AI workloads? At least, this is the impression the landingpage left me with.
I am a bit lost, tbh.
Basically SETI for AI workloads? At least, this is the impression the landingpage left me with.
I am a bit lost, tbh.
There is no distributed execution of a single LLM across multiple GPUs using tensor parallelism here. Instead, it relies on task-level parallelism-breaking down a large workload into independent subtasks that run across separate machines. For example, generating embeddings for a million documents.
https://developer.nvidia.com/gpudirect
And this is because I can't be bothered to dive into older papers, so recent NVIDIA stuff will do.
*: hell my own compiler is moving to multi-GPU on a single computer and networked devices are the obvious next step.
it's a good idea but you often end up with huge tail latencies, slight incompatibilities due to fleet heterogeneity, and a database/scheduler management problem.