You don't have to use them if you want, and I would never bother in some throwaway internet comment where context clues work well enough, but they serve an important purpose in typography that date back to the invention of the printing press or before. Each of these typographic marks signals uses that greatly differ syntactically and linguistically. In another sense, these are building blocks of design, which revolves around subtle yet explicit visual cues. Giving the reader those cues eliminates ambiguity by unambiguously establishing the writer's intent. That in turn promotes visual flow, which is essential when typesetting something hundreds of pages long. If you ever find yourself in this situation, you'll be happy they exist!

As someone who does use em-dashes (and never en-dashes) I don't really disagree. Double quotes in particular are a ridiculously overloaded piece of punctuation. On the other hand em-dashes have some aesthetic appeal vs. hyphens or double hyphens as I would use on a site like this. (Even if I think there is legitimate debate over whether they should have spaces around them or not.) En-dashes? I was on the style committee at a prior company and I don't think we used them because it would have been another thing copyeditors had to fix every time.