I'm not "woe is me" but I was raised in a super conservative family, taught to avoid risk and that a steady paycheck was the ultimate success. It has taken many many years to break out of that mindset and be willing to take calculated risks.
The really hard questions arise when you start talking about what people deserve. Who is to say? We all have a unique frame of reference and experiences that have shaped us.
In the end, the only middle ground we're likely to reach is the mindset of helping people help themselves. To me, the fairness question is more about the opportunity to improve you and yours. Having no opportunities is unfair and must be fixed. The question is always going to be who decides how much is fair?