I doubt you've actually been to burning man. If you have actually been, I don't think you spent enough time looking around or talking to people.
People that shouldn't be there (financially) still find a way to go; it's no different than many of the diversions we pursue in life.
How much do you think it costs exactly? Historically, as little as a few hundred could get you there. Less than the air fare many people spend to fly home for the holidays.