I'd regularly go jogging. People would stop and stare - doubly so if I stopped running and did pushups. In my entire time there, I saw perhaps 3-4 Indians running, and considerably more foreigners. The Pune running club was comprised primarily of people who spent time overseas and expats.
During a conversation with an auto driver, he told me that Indian's don't do "poses" (i.e., yoga) - "that stuff is just for tourists". (I gather there is some regional variation.)
So my guess that the reason India doesn't have a lot of medals is that people just don't care to compete for them.