The number of physical obstacles is less important than the number of legal obstacles.
If it has to go somewhere, everyone wants the high speed rail to go through someone else's house, preferably a long ways away.
If it has to go somewhere, everyone wants the high speed rail to go through someone else's house, preferably a long ways away.
However going through mountain in an environmentally conscious way gets radically expensive in very short order. Additionally this style of construction tends to have its own legally contentious issues. (Dam construction for water management being comparative)
You already pointed out that Los Angeles was a problem. For San Francisco the route chosen ran into problems 30 miles south of San Jose, where they were planning to use existing rails (at slower speeds). And once you're in San Jose, you still have a long ways to go to get into San Francisco.
See https://www.latimes.com/local/california/la-me-bullet-train-... for verification.