To be fair to the OP, I don't think there exists a history of abuse of females from trans men, which strengthens their allegation that the underlying sex might be an important factor to consider in these scenarios. The sex hormones alone maybe just aren't a factor.
It's likely not safe to place trans women into male prisons either though, so I think there ought to another axis than just male/female along which we can segregate, such as violent/non-violent criminal history/tendencies. Come to think of it, that seems almost obvious in retrospect. Put violent offenders together, and non-violent ones together, and then non-violent offenders are far less likely to become harder criminals upon release.