The reason why we have such shitty HSR on the NEC boils down to two reasons: one, the corridor is too congested with commuter rail (particularly in Connecticut), and two, the route was rebuilt to "modern" standards in the 1930s. Rebuilding it again to truly modern 220 mph HSR standards involves needing to find virgin right-of-way in Connecticut, which is neither easy nor cheap.
The Northeast US has a similar statistical density as areas of Europe but looks so much physically different. Take some time on Google Earth and check it out. There is a nonstop blob of suburbs that go almost all the way from DC to Boston. In a proposed train network, most of the population has a long trip to the central station before they can take the train to the other city. They still need a car, and need somewhere to park it. In Europe or Asia cities are more concentrated, and the countrysides are less populated.
Building a center-city to center-city train network in the US is not impossible, but it simply wouldn't be as efficient as it would be in Europe or Asia because of the way the cities are designed.
I very much agree we need a new approach to planning an zoning. We need to focus on that first. It could take longer than a generation to transition away from the suburban model.
Many Americans prefer the freedom of being able to go A to B at any time without needing to wait at a bus/subway stop or hoping the transportation system is still running for the day. The amount of money it would cost to get the entire country walk able or reachable by public transit would be astronomical.
That's probably true (modulo taxis, even if you rule out public transport), but it's just as true when you fly from Kansas City to Houston. Which is why airports always come with car rental, and it works out for those who want a car. If a car is really needed at the destination, arrangements could and should be made to have car rentals available at the train station.
I mean... the shitty high speed rail we have (Acela) does it in an hour and 8 minutes. Making it 55 minutes probably won't change travel patterns substantially. (Although route upgrades to support that is the medium term plan)
And SD - SF should be less than 4 hours, making it faster than a flight (if you count all the time wasted at the airports).
Or, better yet, you use (now non-existing) public transportation to go around.