This might seem like a nit but the term "superhuman" is a VERY strong one to my mind. It doesn't suggest "better than the average human off the street at a particular random task" but instead suggests "better than humans are capable of getting with training, at a high percentile-level".
One of the biggest advantages of LLMs as a tool are that they are generally quite good against a broad variety of things without needing a ton of further domain-specific training. Humans tend to be the opposite.
It doesn't seem like they gave much training to the human annotators they recruited. Whereas an LLM trained on the internet has been trained on a LOT of blog posts + associated metadata. And nobody has ever really bothered figuring out "how would we best train humans to identify gender of blog post authors" - there's very little economic incentive for it. It's not like we generally train people to write in gender-specific ways in school either, so we haven't been formally instructed on potential differences. We'd have to rely on broad-brush generalizations if not given an opportunity to deep dive to try to find more specific tendencies.
But if you pay people to study a big majority chunk of the corpus they're using for this for a couple years, focusing consciously on the post style, contents, and the gender both, and then test them on stuff from the ones you held out... how well could they do?