> Draft MTP <= 2 so it doesn't trip
I am not sure you understand what either of these things do.
Do you think that FA or MTP are lossy?
> Draft MTP <= 2 so it doesn't trip
I am not sure you understand what either of these things do.
Do you think that FA or MTP are lossy?
My experience is that draft-mtp=2 gives 25% improvement in tokens/s but the model is unable to call tools with the right arguments reliably. I have since then gone for smaller quants at draft-mtp=1 and that problem is at least gone so far.
An agent needs to repeat tool calls in the right way, so cannot penalize repetition. This however leads to the agent retrying the wrong calls constantly.
MTP when done correctly is lossless. As you note: it is a draft. That draft is still checked by the original model. Drafted tokens are not accepted if the original model would not have selected them.
If you are having issues, something else is wrong. But in any case, you lack the domain knowledge to be posting authoritatively about it, and I suggest you read deeper into how these things work first.