Wow, this explains an issue I ran into for images from a certain vendor when we used idct scaling for the same reason:

Some encoders (and a meaningful share of JPEGs in the wild) leave zeros or garbage in the unused rows/columns of the trailing MCU. At full-size decode those samples are cropped away, but IDCT scaling mixes them into neighboring output pixels and the artifacts become visible. See https://crbug.com/890745 and https://github.com/libjpeg-turbo/libjpeg-turbo/issues/297

In our particular case we were able to just crop out the edges and it was worth the tradeoff

Thanks, I knew it was much more efficient, but seeing the numbers in the benchmarks make me appreciate this "trick" even more!