A combination of number and method. Death camps, for example, count very heavily, as do things like deliberate starvation.
The intent to exterminate should also be factored in, but that's frequently difficult to determine after the fact.
Of course, none of this matters to people who simply want to make a moral equivalence between every leader of every country through history; my rule of thumb is, when you're feeling morally conflicted about who 'should have' won the Normandy invasion in 1944, your morality is a bit off-kilter.