Are you saying those top 10 cyclists in 2005 weren't doping? Because they all put in times that have never been matched.
Yes, of them Cadel Evans (8th, 11:55 behind) and Óscar Pereiro (10th, 16:04 behind) probably weren't doping. Christophe Moreau (11th, 16:04) has always claimed he'd stopped doping after leaving Festina in 2002 (and this results kind of back this up).
I'm not sure where you get the idea that "they all put in times that have never been matched".
It's true that Armstrong's average speed for that Tour was the highest ever. That's not very useful - the route changes every year so you can't really compare it.
It is more useful to look at times on long hill climbs to mountain top finishes. Unfortunately the best example in the 2005 Tour (the Courchevel stage) doesn't have widely documented times, so we can't compare.
Something like Alpe d’Huez is instructive. You'll note that around the 41 minute+ range you start seeing riders from the 1980s, who can't have been on EPO (since it wasn't invented). It's true that Laurent Fignon was banned for drug use, but the amphetamines he used are completely different to the oxygen vector doping (EPO and variants) used after 1991.
[1] https://en.wikipedia.org/wiki/2005_Tour_de_France#General_cl...
[2] http://www.stickybottle.com/latest-news/fastest-cycling-time...