I half-agree with you, but isn't there an ethical line past which it's wrong to call someone a hero?
I know a good number of people in a relationship, both male and female, who have engaged in some variation of second offense(throwing things or breaking things during their worst arguments with their spouse/partner. But it wasn't habitual and didn't extend to physical contact.
If that's "all" Feynman is accused of, then I don't think it's that big of a deal. I mean, he had a son and daughter who looked up to him, so I am a bit skeptical that he was a brutal wife beater (who tend not to draw the respect of their children).