I considered this and decided to withhold affiliate links for the first couple months of the site's existence. It didn't seem to make much of a difference one way or another.