This is usually caused by having a lot of devices polling your G-mail account. Go to this link: https://www.google.com/accounts/IssuedAuthSubTokens

and remove any that are old/you aren't using. You'll see a speed up.