This is a factor in some places, but a gross over-simplification in others.
There are neighborhoods full of affordable new construction houses not far from where I live. They sell slowly because people would rather live in the popular areas.
There are affordable high density housing options for rent here. They stay on the market because everyone wants their own house.
It's not even about remote work here, as the popular location for office builds and jobs is actually closer to those affordable housing neighborhoods few people want to live in. Being near the office buildings is actually a reason why they're undesirable.
There are some obvious broken housing situations like San Francisco, but I don't see permitting reform as a magic cure-all in every city.