Sliding Tile Attention: A New Method That Speeds Up HunyuanVideo's Outputs by 3xold.reddit.com·2 pts·zhisbug·1
FastVideo: a lightweight framework for accelerating large video diffusion modelsgithub.com·110 pts·zhisbug·24
MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Servinghao-ai-lab.github.io·2 pts·zhisbug·1
Consistency LLM: converting LLMs to parallel decoders accelerates inference 3.5xhao-ai-lab.github.io·461 pts·zhisbug·98
Throughput Is Not All You Need: Maxing Goodput in LLM Serving via Disaggregationhao-ai-lab.github.io·5 pts·zhisbug·1
Fastchat-T5: 4x smaller but more powerful than Dolly-v2, commercial use readytwitter.com·7 pts·zhisbug·1
Alpa: Auto-parallelizing large model training and inference (by UC Berkeley)github.com·7 pts·zhisbug·1