I completely agree. Pronouns, just like names, are simply references. It's how you refer to someone.
You wouldn't assume you could look at someone and guess their name. Why assume you can guess their pronoun (from their appearance or name)?
NOTE: name+pronoun references != identity. Gender identity is a spectrum, and can't map 1-to-1 to low-bit pronouns.