While I don't disagree, I will note that most people don't think twice about giving up control to bus and taxi drivers.*

I think we trust humans to make a reasonable decision in a trolley-problem scenario (rightly or wrongly). Or rather, we trust the human we're in a vehicle with to value their own life, and thus our own, more than those outside of the vehicle in most scenarios.

I expect there is research to investigate this, though I wouldn't know where to begin looking.

*I've definitely had a few bad drivers, of course.