Is it? The amount of deaths shouldn't be compared to 0, it should be compared to a counterfactual where the indigenous people were in charge. Otherwise you run into the "doctors with most patient deaths" problem. It's possible that the doctor is objectively bad, or that he's receiving the sickest patients. It's plausible for instance, that areas with local rulers who are brutal and despotic are easy to conquer/colonize by an outside empire, because the brutality/despotism means it's easy to assemble a local coalition to depose the current regime.
For example Russian excess deaths all happened over quite a short time span. Does it mean their intensity of badness was higher?