How are you ever going to achieve high speed if you are stopping every 10 miles for another station? There appears to be 8 stations in Baltimore alone.
But then the bizarre/historic track geometry ruins the fast service...
Rerouting so far has been a headache; in New Haven-Boston where the tracks are so curvy new tracks have to be built, the FRA proposed going through entirely different population centers (the geography of the existing coast line is very challenging). This has resulted in an interesting fight where people want the train in their town, but also do not want the impact of newly constructed tracks through their town.
They really can't.