I am sorry but I am specifically speaking of cases where the phenotype contradicts the genotype. For example, there are cases where an individual has an XY karyotype but develops female traits due to a mutation in the SRY gene.
The word "sex" is ambiguous, do we speak of phenotype (sexual traits/appearance) or genetic characteristics? And if you categorize people by their appearance, how do you categorize people with atypical genitalia (~1/5000)?