> It's likely not safe to place trans women into male prisons either though, so I think there ought to another axis than just male/female along which we can segregate, such as violent/non-violent criminal history/tendencies.

Male-on-male violence is a problem for the male prison estate to solve within its walls. It doesn't make any sense to put women at increased risk of male violence to protect men from male violence. Women don't exist as some sort of violence shield to protect men from each other.

Men's prisons already have safeguards in place for vulnerable male prisoners, such as paedophiles, ex-police officers, ex-gang members and so on - putting them on separate wings of the prison, moving them to a different category of male prison (which is in practice similar to your suggestion of violent/non-violent separation). If men who call themselves women are at elevated risk from other men, the same safeguards can be applied to them. There is no reason whatsoever to house them in a women's prison.