I would agree that it might be a disadvantage for women, but conversely I would just make an effort to ensure there was _someone_ else with us, male or female. So instead of drinks 1:1 outside work, or a 1:1 meeting, It would instead be three people which mitigates both parties liability.
I am a senior engineer, not a manager, but if I were a manager one on ones would be necessary, I would attempt to have HR in each 1:1 as an ass-saving measure anyway.
But then again, I believe I would be a poor manager so I don't wish it on anybody.