Anyone who lives in a Japanese house will attest to the awful feeling of waking up in a freezing house in even a mild winter in Tokyo. Or the murderous hot of summer that it has its own word for trying to escape it (避暑). The poor insulation means you can spend myriads of yen and never feel comfortable inside.

Incidentally, this is also a reason my colleagues wanted to go to work. Offices are often better setup than homes. (At least this was their excuse to stay at work)

Perhaps that's true on average, but there are some very cold places in Japan and the housing is the same there as I understand it: built to last 20-30 years.

For example https://en.wikipedia.org/wiki/Asahikawa#Climate

Less than northern continental climates sure, but you're ignoring the jet stream. Tokyo is considerably colder in the winter and hotter in the summer than, let's say Monterey at a similar latitude.