Brushing vigorously can erode gums, exposing the sensitive parts of teeth. It's very important to brush gently (but thoroughly); not back and forth, rather up and down. Flossing isn't particularly good for gums, either, so best to do it sparingly.
We probably should floss because it's probably helpful, even if the evidence is weak, it has a plausible mechanism of action, it's low cost, low risk, and teeth feel good after a proper flossing.
A big issue is many people have poor flossing technique though, which is (probably) not great for gums and may actually cause problems. Flossing shouldn't hurt and you should talk to your dental hygienist about proper (recommended) technique. Also have him or her actually floss your teeth if they don't already during a cleaning (mine does) so you can get a feel of what the recommended technique feels like. Also try different types of floss. My hygienist can floss me just fine with the "normal" type floss but I can't replicate that for some reason, I need to use the "glide" type floss.
[1] https://www.washingtonpost.com/opinions/the-evidence-for-flo...
Source: married to a dentist.