It makes a lot of sense to connect San Diego/LA/Sacramento/San Francisco.
It makes a lot less sense to try to connect Merced, Bakersfield, Fresno, et al. People there like to have cars, like to drive, there isn't a lot of traffic. Once you arrive in those places, there is very little transit infrastructure. You basically need a car. And they're far more centred around ag or industry, so more reason to have commercial / truck traffic and a lot less for just passenger cars.
Meanwhile, there are over 100 flights a day between LA area and SF. Meanwhile, Merced has 2 flights a day to LA on a tiny prop, Bakersfield has 2 to SF, and Fresno around 5 a day. There aren't any flights at all between Bakersfield/Sacramento/Fresno/Merced.
Whereas SF/LA/San Diego make complete sense to have a train station with plenty of transit options to get around once you arrive.
(To get an idea of what I'm talking about - traffic on I-5 is so heavy, we would often take 99 instead, when going between SD/LA and Sacramento. 99 is 2 miles farther than I-5.)
San Diego/LA/SF/Sacramento is one of the few markets in America that could reasonably support high speed rail. And it's sad to see it being strangled in the crib.