So if Bob wants to call himself Brenda, wear a frock and make-up, and take drugs to grow breasts, then that's fine and he shouldn't be fired from his job for doing so. But this doesn't give him access to women's spaces, and if any of his colleagues don't want to refer to him as "she" then they shouldn't be censured for doing so either.
This stance also protects LGB who may want to organize same-sex groups, such as lesbian speed dating or gay men's saunas, without having individuals of the opposite sex imposing themselves for self-identity reasons.