I feel as though this discussion has more to do with the control over the discussion than with the content of either phrase.
If you want an explanation better than "Oh well, you know", you might start with the criticism section of its Wikipedia entry, at https://en.wikipedia.org/wiki/All_Lives_Matter#Criticism .