Sure, but he doesn't say that self interest is or should be the only interest, only that markets and economic activity work best when self interest and the interests of society align.
Adam Smith wasn't a free market fundamentalist in this respect, much as the fundies would like to make out. He was very much in favour of regulation.
I like this:
"..The proposal of any new law or regulation of commerce which comes from this order [businessmen and traders], ought always to be listened to with great precaution, and ought never be adopted till after having been long and carefully examined, not only with the most scrupulous, but with the most suspicious attention."
SOPA and ACTA anyone?