I managed to find a decent looking study that somewhat supports my position [1]. Here's an interesting quote from the abstract:
"many (40%) were hesitant to consent to the release of their personal profile information; and (6) many (36%) ex- pressed concern with the use of SSO on websites that contain valuable personal information or, conversely, are not trust- worthy. We also found that with an improved affordance and privacy control, more than 60% of study participants would use Web SSO solutions on the websites they trust."
[1] https://cups.cs.cmu.edu/soups/2011/proceedings/a4_Sun.pdf
This is a very important discussion -- I'd love to see some papers that study the benefits of a single login / profile, rather then the negatives.