I could see a changed "Manhattan skyline" as a significant impact for many people. For a while I could see the section of the pentagon hit by the airplane outside my window, but there is little lasting visible impact on the structure. Internally, they did a fair amount of renovation for example added some glow strips on the floor pointing to the exits which seems like a good idea, and gas makes which seems less useful (and more creepy). You can even go on tours of the pentagon which seems risky, but I suspect the benefits outweighs the risks.
However, that's all fairly long term effects. As to being up close and personal, a coworker of mine was in a conference room when 1/2 the people at the table where taken yet he had no problem going back to work at the pentagon. Sure, it's a target, but numbers suggest being a truck driver to be a far more dangerous profession.
When it comes down to it terrorism is simply not that effective. Several paces have had sustained terrorist attacks for long periods of time and people don't react that much.