Cost of healthcare has skyrocketed. As has housing, as you've mentioned. Those are much bigger expenses than groceries and clothing, so those lowered costs are not offsetting the biggest jumps.
Education as well. Healthcare, education, and housing (near a good job) are the tickets to a stable middle class life and they're out of reach for many.
Don't forget, for most parts of the country, a car too. The fourth guy is being limited by his lack of a vehicle.