I've wondered that too. My inexpert, unresearched guess is that people are scared of darkness, because we don't see too well. Hence black is a scary/bad color, and blacklists are the things to avoid.
Brightness is good - you can see, it's usually warm, etc. Hence whitelist.
But like I said, total off-the-cuff guess, take it for what it's worth.