It's not remotely controversial in Scotland - estates effectively have legal requirements for the numbers of deer they need to shoot each year to keep the population even vaguely under control (in fact, many campaigners complain that estate populations are allowed to keep deer populations too high to improve the commercial hunting.) This is generally accepted because, as you say, they have no natural predators. Is the situation different in England? I didn't realise it was.