Being outside of the workplace does not make a power relationship that is akin to slavery OK. (You could claim the woman is free to leave, but there are implied threats against doing so, such as an asymmetric implied threat to reputation, with email documentation, at a minimum).
Yes it is true that context matters. However, I think it's safe to say his ex-wife who shared the quote has more knowledge of the context than we do... and again, slavery is not an indicator of leadership to aspire to. Nor is paying massive rewards to such a person.
Edit: what part of this do you disagree with?