Personally I wouldn't want to rely on telephoto lenses. They feel kind of disconnected with the subject and are more creepy than using a short focal lense.
I returned a 70-200 after trying to lug it around for 4 hours. It's just too tiring and too silly for general use, and it's certainly not inconspicuous if you're going to be in public.
I have a 24-105 f/4L that I use for day time and a 35 f/1.4L that I like to use at night. I do own a few other lenses but the two mentioned make up 99% of my usage.