Wrong. It's most definitely a thing in quite a few mid-sized Japanese cities. Here's a compilation video taken at Fukushima Station (https://youtu.be/K-wkX3vFU_A?t=403) on the fastest 320 km/h line for example. You can check for yourself that the station is dead smack downtown in a city of 275,000.
At the world level it doesn't tend to happen because existing developments in cities usually strongly inhibit the construction of straight rail alignments that can support high-speed operations. But that's not a problem in the Central Valley – they're blessed with excellent existing rights-of-way.
If CA HSR can go through the cities in the Central Valley and still achieve an average speed of 250 km/h, that's well worth it.
Your business class traveler from LA/orange county to SF bay is already well served by the many airports in both metro areas.
the HSR is about connecting the rest of the state to economic opportunities in these large metro areas
I took a 700mi HSR in Japan that was probably on the very far end of being competitive time wise with flying and was still great. 5hr train vs 2hr plane segment, but all-in door-to-door travel times were comparable (5h45m vs 5hr).
Train 5hr45m door to door with majority of time sat in a comfy quiet train with big comfy seats and high speed internet. A flight which is 5hr door to door is mostly a ton of hurry-up-and-wait with small blocks of 30-90min here or there you can read a book.
If you're going to stop throughout the rest of the state you just want normal rail running at 100-120mph.