Yes, and they're right. People change genders or aren't part of any; there's gendered dress codes which societies enforce (or people beat you up, you can't get a job, etc); and so forth. Hell, Alan Turing was basically killed by the state for not adhering to gender norms.

With the concept of "gender," people refer to a huge social infrastructure. With "biological sex," they're trying to refer to something else.