It is actually a wealth transfer. Think of houses as denominated not in USD, but in labor hours (let's say median wage). If the house price appreciates in labor hours, that is actually a wealth transfer from those performing labor to those holding on to capital (and vice versa).

In reality it's a little bit more complicated since the financing cost of the house has to be considered (principal + interest over the lifetime of the loan), instead of the notional value of the house (and refinancing throws another wrench into the equation).

But as a first order approximation, if houses got more expensive in labor-hours then that's a transfer from labor to capital (and a "hidden" tax on anyone holding USD).