The median Bay Area household cannot afford to buy a house, yes, but that does not mean that those buying houses are significantly wealthier. The Bay is far, far richer than you can imagine.
Buying a house for $1M is firmly middle class status. True lower class people simply cannot afford to live in the Bay at all.
To be in the top 1% in the Bay Area you have to make close $1M per year.