It probably depends on the stage of your life. I had a house with backyard when I was single and didn't use it much.

Now, I have family and kids and totally enjoy having it even though I only get to use it 4-5 months out of the year: - kids spend a lot of time there on the trampoline - my wife has an organic garden and beautiful flowers - my dog chases rabbits and squirrels - enjoy being outside without having to go to a park (reading books, playing with my kids) - family gatherings, BBQs and etc

There's no way I'm going back to living in an apartment or even a townhouse with no backyard unless I really have to...