If everybody in the UK moved to where houses had average cost, everybody would be able to afford a place to live. If they could get a job, which they can't, which is (a large part of) why they don't do that.

That's the problem with averages.

The high cost of housing is caused by demand outpacing supply for a very long time. It's expensive to live in London because more people want to live in London than the housing stock will accommodate, and no amount of redistribution will change that, except to the extent such a policy makes it drastically less attractive to live in London.

The exact causes of the supply shortfall is subject to some discussion, but that the shortfall is the proximate cause of the high cost isn't.