I don't think it's been alleged that he has been deliberately leaking info to China. But making decisions for the benefit of a single customer acting as a Chinese cutout to circumvent US export rules risks serious legal or reputational damage for Nvidia. And overall means he could be making other decisions not in the interest of Nvidia shareholders while under her influence, consciously or unconsciously.
Nvidia is going to be a political punching bag facing endless subpoenas and hearings because of him. Didn't even have to wait until the new Congress - the lame duck Congress is already getting started so it's not a one sided partisan Republican or Democrat thing either.
Plus, as others have pointed out, the Nvidia board appeared to have been preparing to back him up and ride out the story after his first carefully written apology/statement. But that seemingly changed after news of the second affair broke which means he may not have been candid with them and made the breach of trust unsalvageable. Unfair or fair doesn't really enter into it, it's not his company anymore, he only owns ≈4% of it. Nvidia board/shareholders would have wanted to keep him running things if at all possible given his track record.
Evidently some straw broke the camel's back, but we don't know for certain whatever it's something that's been revealed publicly so far. It's high profile news because of Nvidia's size and prominence in the AI industry but he's far from the only founder to be pushed out of the company they founded because of a personal scandal.
The political argument doesn't really make sense to me considering Trump bent over backwards to defend him, right up until minutes before he resigned. And the admin seemingly avoided intervening for months after it became known to U.S. intelligence, even adding him back to the China trip after he was originally left off the roster.