My issue with Feynman, the one I cannot get past (no matter how much I like his lifestyle and intellect), is the work on the Manhattan Project. His defence was, essentially, that a war was on, and the US needed the same capabilities as other nations were developing. However the resultant catastrophes at Hiroshima and Nagasaki just negate that for me. He didn't order those bombs to be dropped. But he is partially responsible for those deaths, whether he foresaw them or not.
Incidentally, I've not read much about his womanising - perhaps I'm not reading between the lines?