Yes, lifestyle inflation is a real problem. But it's certainly not universal. Plenty of people maintain high savings rates instead of consuming excess income. The discipline required for early retirement communities isn't very different from that required for earning to give.
That being said, I have found that it helps to maintain a diverse social sphere. The majority of my friends make far less money than me, so it's easy to match my lifestyle to theirs (in fact, a bit better) while having plenty left over for donations.