I have spent some time in the San Francisco bay area and a lot of time in New York City.
Looking at BART and Caltrain schedules - the last Caltrain train leaving San Francisco is three minutes after midnight. The last BART train leaving San Francisco leaves at 1041PM! The BART website says they will add slightly later trains next month, but still - it was like this before Covid too.
In New York City, PATH and subway trains run all night. Metro-North, LIRR and New Jersey transit trains run fairly late as well. It's a larger city, but many people commute 40 miles or more every day, and it's no big thing - get on a train in Bay Shore, Suffolk, New York in the morning, and in a little less than an hour you walk out of Penn Station, over 40 miles away. If you want a faster commute, then live 35, or 30, or 25 miles away - still plenty of room for one family homes and lawns if that is your thing.
I have taken the BART from SFO into the city during the day, and that worked well enough when I did it, but expanding BART, Caltrain etc. could solve a lot of the Bay's problems of insane rent for a small place in a neighborhood that is not that great.