That would very clearly negatively affect the opposing party.
You first have to define the opposing party. Are you speaking of the slaves or of the other humans who oppose slavery? Because remember, part of the justification is that slaves and unborn babies are not fully human, and thus cannot be the opposing party.
If you have slavery, obviously you're not going to care much whether the slaves oppose it.
Slavery kinda influences lives of slaves. Maybe.
But slaves don't get to vote, so it's a moot point.