You really have to be specific about where you're collecting your anecdotal data.
If you live in Texas or much of the Southern US, being outside during the summer is nasty. As though the heat isn't bad enough, the mosquitoes are positively vicious. When I lived in Texas, we spent most of May-September indoors. It was great to have a big house there.
Likewise, you need to consider how snowbound many portions of the US are during the winter months. Being outside from November to March in Minnesota is a great way to freeze, but gardening isn't really possible.
When I lived in California, the weather was beautiful and the bugs were few. We lived in smaller places and it didn't matter since we were outside so much of the time.
It's a big country with many different climates.