I think the discourse of AI doomers is more specifically about AI taking control and acting against us, not really about AI posing risks in general.

I do think AI poses huge environmental and societal risks, including for reasons in the spirit of what you say. That's not being an AI doomer IMHO. That looks way more reasonable to me.