The problem is that the overwhelming majority of spam has a false return address, and the technologies to verify that mail came from the server it claims aren't yet widely enough used to be reliable. The bounces create an extremely large amount of traffic for servers unrelated to the spammers.

Besides, many servers do behave in the way you specify. Before I had the sense to disable catch-all email addresses on a few domains I owned, they were all being flooded with bounced spam, much of which my spam filters didn't catch.

Blah. Everything's a mess.