I think you probably don't have a good sense of just how much energy walking and other continuous small movements burn. There have several studies that showed that even different amounts of fidgeting can result in significant differences in energy usage, which correlated closely with levels of obesity.

"Overweight people have a tendency to sit, while lean ones have trouble holding still and spend two hours more a day on their feet, pacing around and fidgeting, researchers are reporting in findings published today.

"The difference translates into about 350 calories a day, enough to produce a weight loss of 30 to 40 pounds in one year without trips to the gym - if only heavy people could act more restless, like thin ones."

....

"The director of the study, Dr. James Levine, an endocrinologist and nutritionist at the Mayo Clinic, said the findings offered hope to overweight people, suggesting that relatively simple and painless changes in their daily behavior, like making an effort to walk more and ride less, could help control weight."

Source: http://www.nytimes.com/2005/01/28/health/28weight.html

I picked this one because it's the NY Times and Science, but there are numerous other sources available via Google.