Considering how competitive physics graduate programs are at top schools it is a shame to hear that so few are pursuing careers in physics and some appear to be calculating in advance their move to CS. I love CS but if I had a passion for physics I would be pissed to see someone that society, schools, mentors, peers have invested in to do physics go be quant at an investment bank. It is not that working for a bank is intrinsically bad, more the case that individuals should consider the opportunity they may have been taking from others in pursuing a specialized education they subsequently don't use.