The charter is meaningless weasel words and is used just as often to curtail the rights of Canadians as it is to protect them.
The situation is exactly "tails I win, heads you lose" but in favour of whoever has the majority of MPs. They can take illegal action to thwart political opponents and face no consequences. Just as how police officers should be punished when they break the law (but often aren't), those who hold political office should be punished when they step outside their bounds.