The point is, in web design there are a lot of different aspects that have to be balanced. One of them is accessibility & speed, an other one is visual design and identity. The choice of a web font over one of the ~6 standard fonts affects both of these and there is a choice to make.

This choice can be: I use a web font because this allows me to convey 80% of my visual identity with the text, I can put the text (and content) as the central element of the page and include no images on the page except for a small logo, leading to a fast loading page. For this, I accept that the user has to wait a little bit until the text is displayed when he/she visits the page the first time and the font isn’t cached yet. On a decent internet connection this might not be noticeable but on a bad 3G connection it might take the page load from 4s to 6s. Do the benefits outweigh the risks/disadvantages?

I do agree though that this decision is often not made consciously and a lot of times a web font is slapped upon a site just because we can. Way too often, visual designers create pictures of websites in Photoshop with no idea about the implications. On the other hand, I have seen that the rise of web fonts has made designers shift away from creating designs that are basically a frame with the visual identity where they slap in some random content. A lot of sites now treat the content as the main element of the site, reducing all other elements to a minimum. I would argue that this change greatly improves the quality of web sites and that the rise of web fonts is one of the main drivers of this change.