Oh? So you don't think sex is binary but want to discuss neurobiological differences "between the sexes"?
Which ones do you want to discuss the differences, between?
Which ones do you want to discuss the differences, between?
Biological sex isn't actually a spectrum (it's not one-dimensional), but using that analogy, it's bimodal, not binary.
Only how (or if) one chooses to include those few who don't fit these categories is fluid.
In other words, there's some uncertainty at the margins, but in the vast majority of cases, biological sex is clear.
If you can't clearly define a category, any intrinsic traits of that category are not ascribed clearly to individuals. Any uncertainty, even on the margins, makes the definition uncertain. Clear definition is a lack of uncertainty.
The fallacy is the argument that two states or conditions cannot be considered distinct (or do not exist at all) because between them there exists a continuum of states.
In other words, your reasoning is a textbook example of this fallacy.