It's notable that this tree was probably a few hundred years old. That's not especially old by European or tree standards, and this tree was of no real historical significance. But it's older than any of those structures you describe.

The brutalist structures might one day be beloved. Perhaps a few centuries from now. They're certainly built for it; one problem with them is that they're hard to demolish. (And often lousy air flow, which is hard to retrofit because the walls are literally artificial rock.)

Ugly things can get some sentiment once you see it as "yours". Time can do that. Notions of beauty change, and things go in and out of fashion. Even trees have been seen as ugly at various points, when what people wanted was to tear it all down and build things.