It's not that those people groups don't matter, it's just that if it wasn't COVID-19, it could have been one of a dozen of other common ailments with elevated CFRs for those populations. My Nana beat cancer multiple times, but ultimately succumbed to a minor cough that turned into pneumonia.
Eventually time gets the better of us all. I don't think that's an insensitive statement, just pragmatic/realistic.