HNHacker News
TopNewBestAskShowJobs

fergus00

1 karma · joined October 28, 2010

submissionscomments
fergus00··on Reverse-engineering Nvidia's CUDA-checkpoint for faster cold starts
https://blog.doubleword.ai/fast-sglang-starts yeah, this is part of the idea. If you get cold starts down to seconds or subseconds, then you can run many models multiplexed on the same GPUs