I'd think anyone hiring a personal assistant would be on the advantageous end of a financial power imbalance. Are you just saying a young man wouldn't be hired to clean a house so it's embarrassing when a woman cleans a house? I think the important thing is to pay cleaners enough and make reasonable requests, not refuse to work with them. (I don't have a cleaner, if it matters.)