I'm want to agree with you, but I have a question.

Binaries tested against what? Obviously the testsuite that upstream uses, but every program will have dynamic dependencies into the distro. In that case wouldn't the upstream maintainers have to test every version or every distro to make sure it works for you? That seems time intensive (not infeasible, but it would become chaotic very quickly when we multiply the number of distros by the number of apps people want).

I'd also presume that the upstream maintainers include their testsuite with their source, so nothing should prevent the downstream distro maintainers from running the testsuite before shipping (I have always assumed they do, I hope I'm not wrong).