One doesn't need to keep anything in mind if one just has two different clients for the two user groups. And there are IRC clients, such as thelounge or irccloud (ok.. more than just a client), that offer things like inline images/audio, link preview, etc. (and wouldn't be hard to add missing things there). On the protocol level they just send urls in the irc messages, which falls back nicely for the other user group. I send images, pastes, etc. all the time on IRC it's just I don't want my client to render any of them inline - I want to decide if I look at something or not, while you want a client that does render everything inline for the most part.
PS: My spam filter judges html emails rather harshly :P