An Open Field Guide to Inference Engineering

Great guide, thanks.

The Case study for me was the most interesting part; it shows that you really have to check all the different options to get to your optimal serving stack, and that there's no single best-solution to your configuration - each workload, traffic and hardware require their own unique mix of engine+configuration.

BTW, the link doesn't work - I think you submitted https://docs.modelplane.ai/main/guides/inference_guide instead of https://docs.modelplane.ai/main/guides/inference-guide.

We (the Modelplane OSS maintainer team) wrote an open guide on inference engineering based on our experience running open-source frontier models at scale with references to some of the best reading materials shared in the industry. Check it out, ask questions, discuss and contribute!