BART needs to just go around the Bay Area entirely in a loop, or Caltrain needs to become more efficient. Hopefully electrification will help in this regard.
BART is going to San Jose. It will go to San Jose Diridon thus completing the loop. It will take longer than everybody likes and probably cost more money than was originally budgeted, but it's going to happen.
Caltrain presumably will remain the fastest option from San Jose, at least if regressive enclaves like Atherton don't delay the improvements for the high-speed rail project.
Getting the fiefdoms to coordinate is the Bay's eternal transit problem, however.
OTOH Atherton may well be one of HSR's smallest problems.
I support the project, but part of me wishes that the money could have gone to massive improvements of Los Angeles, Bay Area, Sac, and smaller cities' transit (with density bonuses, value-capture schemes, etc.). No way that wins a statewide ballot though.
In the end, the Caltrain improvements as part of the project will help a lot, which I think include more double-tracking as well as electrification and grade-separation.