I am Indian and I want them to stay. I would go so far as to say that colonial era structures are the only beautiful buildings that have been built in India in the last 100-200 years.
There is no Indian architecture in the last 100-200 years because of the brutal british colonialism.