I think part of the controversy is that Stallman is known for being an uncompromising absolutist on open-source. So people assume he has the same attitude towards his terrible ideas.
Stallman’s ideas on open source are well thought out, and vigorously defended for decades.
His terrible ideas are neither well thought out, nor extensively defended.