I know this doesn't retrain, but I wonder if approaches like this plus quantization could get back any "lost" quality with some post training.
It's great to see, and to know in one's mind how much likely performance and cost will be better in the future.
I know it's fun to work on, but I also want to say THANK YOU for developing open source.