Even in the case of gmail, which is the best case, you can not be sure the person in control of the Google account is the same as the one that used that gmail address to create an account on your site. The person might have got their Google account hacked, they might have used a shared gmail account to sign up, your service might not have been properly verifying email addresses when they signed up, etc.
But yes, to be fair, if you have email-based password reset functionality, it is not really an additional security vulnerability.