>Given it's the empire responsible for the most human deaths in history, it's hard to argue

Is it? The amount of deaths shouldn't be compared to 0, it should be compared to a counterfactual where the indigenous people were in charge. Otherwise you run into the "doctors with most patient deaths" problem. It's possible that the doctor is objectively bad, or that he's receiving the sickest patients. It's plausible for instance, that areas with local rulers who are brutal and despotic are easy to conquer/colonize by an outside empire, because the brutality/despotism means it's easy to assemble a local coalition to depose the current regime.