I think slavery is the most damning part. Without it, slave trading wouldn't be possible. It's even debatable if plain slavery hurt people more than slave trading.

Every historical figure I can recall that fought to abolish slavery, didn't target slave trading specifically.