Apparently we’re in the minority together. Cynically (and practically) there’s also the legal side to think about. If you cause a person to be killed by a trolley, you will at minimum be sued into oblivion. If you do nothing no legal case can realistically be levied against you.
Edit: a fun part of the trolley problem not really explored here is how inconsistent most people’s logic gets when the outcomes are effectively the same but the actions are slightly different. For example, what if instead of pushing a button you had to shoot the person on the tracks? What if you could kill someone today who is going to shoot up a mall tomorrow? More fun, what if you could push a button to kill a millionaire and redistribute their wealth to save 100 families in subsaharan Africa?
Logically, as soon as you go down the path of actively choosing to kill a person to achieve some “greater good”, you’ve really thrown an awful lot of morals out the window.