Say my marginal dollar is taxed at 40%, and I buy a house valued at $500,000. That value includes not just the house, but the value of the tax deductions. Over the first 10 years at 4% interest, I pay about $17,500 in mortgage interest, which make the value of the tax deductions about $7,000 per year, or $580 per month. If you get rid of the stream of tax deductions, you're lowering how much the house is worth to any buyer who has labor income and will be financing the house (i.e. almost everyone).
Indeed, for almost all houses (even in the Bay Area), people count on the tax deductions to determine how much house they can buy. If you can afford to pay a net $1,800 per month, for example, you can buy a $500,000 house (with a $2,400 monthly payment) given the roughly $600 per month in tax breaks. If you get rid of the tax deduction, you can no longer afford a $500,000 house, and neither can all the other people who previously had a $1,800/month budget. In the long run, that drives the value of the house down by the amount of the lost tax deduction.