We don't know his relationship with his wife, what their stands on cheating even stands.
You can argue its an image issue, but thats a problem of the press and marketing and not that he might have cheated.
His wife is an adult and is for sure not poor, so its not a situation were she has to stay with him.
I do believe it either has to do with the whole spy story or he did something he really shouldn't have.