By your logic, they don't particularly contribute to scientific productivity/advances: it's not like you can use stable diffusion to generate an architectural diagram.
GenIA for images and videos is only really used to copy artstyles, make fake menus and create misinformation so wouldn't you agree that eradicating this part would not only preserve your described "useful applications" but also allow providers to allocate more resources to "useful" and "scientific" AI?
I know that local models have become decent enough that this isn't really doable, but I'm only proposing this thought experiment because you seem to think people are saying this should be all or nothing. But the truth is, if you get a genie to wipe away just diffusion models, the entire internet gets much better (or at least back to regular levels of bad), no one loses anything, and a whole lot of people stop complaining/campaigning against the productivity gains in your field.