Throughput Is Not All You Need: Maxing Goodput in LLM Serving via Disaggregation

New work from the vLLM team that disaggregates prefill and decoding to maximize goodput (throughput subject to latency constraints) in LLM serving