In Japan and many other parts of East Asia, it rains most during the hottest months of summer. You don't wear jackets when it's 30+ °C and humid outside.
But after living in the region for many years, I agree people also tend to be a bit irrationally afraid of the rain.