Those would be the psychopaths and sociopaths, but I don't think you're among them. These folks basically lack the relevant social emotions altogether, so they end up being quite materialistic and self-centered. They can and do figure out that other emotions somehow exist and that they're socially expected to show them, but they have no "hardwired" intuition for their affective implications, so they'll have a whole lot of trouble negotiating constructively with others. They're deeply prone to thinking that everyone else is just like themselves, self-centered and quite willing to screw others in pursuit of their own goals.

I.e. their affective impairments do have significant implications on the theory-of-mind/'cognitive' side of things, but those implications are very different from the troubles of someone on the autistic spectrum. In a way, they're the very opposite!

(In fact, if you're somewhat on the spectrum and basically aware of what makes psychopaths tick, you can even have a lot of fun spotting them and interacting with them. Their attempts at superficial charm will most likely fall flat, while their lack of deep integrity and engagement will be blatantly obvious to you - but they'll probably be assuming the very opposite! So it's quite easy to figure out what they're going for and anticipate their moves.)