There are a lot of bad angles to this, here is another one: What happens to people's social security they've build for the years they've been on H1B? (Or J1, etc. before that)? It's technically gone...
At least for now, who knows if that'll change down the road.
Some further googling led me to "totalization agreements" and then it's per country.
It looks like India (for example) may not have such an agreement, but say Austriallia has it. That to be said, it's all news to me, so might be completely wrong here