Hacker News Reader

I'm working on a variation of MTP that recovers PP TPS (back to the same as with MTP disabled), keeping most of MTP's benefits to TG TPS.

Will propose a patch back to llama.cpp or provide it as a fork.

Thank you!

If you want to run the PoC locally to replicate:

Clone the masked-nextn-skip-catchup branch:

Run llama-server with at least --n-gpu-layers 99 --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 --no-cache-prompt.

I used Qwen3.6-35B-A3B-UD-Q2_K_XL MTP:

I tested on a L40S (using Modal).

About the parameters:

--parallel 1, see Code review findings:

--no-cache-prompt, see Known limitations: