If my house goes down in value but the trend is roughly general then I don't care indeed.
It matters to me for many reasons I doubt you find sickening or sinister. It matters because my family might want or need to move for work. It also forms a major part of my retirement, and how I will feed and take care of myself in advanced age
It makes perfect sense that it has financial value to someone else, because it would take a tremendous amount of material and labor to re-create if it burned down.
You buy a house, use it to save on rent and protect your retirement from inflation. Come to old age, you reverse mortgage or sell and go back to renting. Many people need to downsize anyways as their energy and capability decreases.
It is pretty unrealistic to expect that everyone can die without breaking into what is their largest expense in life.