The App Password you created for FastMail to access your Gmail account over IMAP cannot be used for anything other than mail/chat/calendar. Additionally, you have 2-step auth enabled so any access to change your account settings would require that.
Is the need for 2-step auth for account changes a documented feature? I'm asking because I couldn't find anything pointing to that and, if so, it could solve my initial problem.