By this reasoning, we should destroy all buildings that were constructed with slave labor; and all jewelry with gems acquired by similar means.
Still, if the human remains are an issue, well - in numerous ritual burials of rulers, there are remains of servants or concubines. In most cases, we can assume it was not consensual.
In this case - should we cremate (or otherwise destroy, according to our contemporary customs) these human remains?