Then you're part of the problem! The attitude is, if it doesn't fit the "Hacker News" relevancy stereotype then flag away. Have you not wondered or even humbled yourself to consider that perhaps your own view is more short-sighted than the posters? I guess flagging in general takes an error ratio into account but it's my experience that it's far too swift here on HN, meaning more weight is given to the flaggers than the posters.
Honestly though, thank-you for asking and posting your actions here for discussion! That, in my opinion, is a lot better than the countless silent flaggers everywhere flags exist.