This is kinda nuts.

Can this mostly be fixed in software, or are M3 users basically stuck unless apps work around it themselves?

Going from 10 to 24 tok/s just from a workaround is a pretty crazy jump.