In many ways my life has been just fine: steady well-paying employment, wonderful wife and kids, etc. So as with you, being (probably) somewhere on the spectrum isn't correlated with a crappy life.
But...
I've also had some unusual personality traits for my whole life, and I've really beaten myself up regarding them. Once I had a probably Asperger diagnosis, I felt like a tremendous weight was lifted. I no longer felt like I was uniquely messed up in those way. And maybe the subconscious logic is that if I'm part of a cluster of people with these idiosynchronies, maybe there's an underlying physiological cause that's somewhat out of my control, and therefore I'm less morally culpable for my atypical social failures.