No. I care about typography. You left out the critical part of this sentence: "it's not important if the default font is used".
Now I understand that you also say that default fonts are not necessarily well designed. On my side, these default fonts have looked good on every device I have used, and I'm more often annoyed by arbitrary font settings that look good to the designer who happened to design the web page I'm reading.
> The user is always in control. They can always choose what to load and what styles to apply
In practice, I'm not going to design each page I visit. I'm blocking web fonts by default, but that sometimes triggers a bad setting where the font that is used as a fallback looks very bad. I guess that happens when people assume that the web font will be loaded and don't default to (sans-)serif in their font-familly CSS property.