My thought is that punctuation "terminates" a sentence, and so the mark that's further to the right is "terminated" later, therefore it more represents the overall sentiment. The left punctuation mark is like a modifier; it's an adjective before a noun.