The way I see it, if those lonely elderly people's days are brightened by being given a flower, who cares what the intentions of the flower-giver were?
Under other philosophical frameworks, intentions do matter, not just consequences. For a deontologist, for example, thinking of Kant's categorical imperative.
My point here is that "who cares what the intentions are" is not really, at all, such an obvious thing. And there's a wealth of philosophy that's non-consequentialist. I honestly invite you to read a bit more on these kinds of ideas because they might genuinely give you a new perspective on morality.
There's a comparison to be made to choosing to stay in the matrix or unplug yourself from the illusion.