I’m curious you could still ride it by leaning…
This is the same principle of balancing an upright broom vertically, handle first, on the palm of your hand. When the broom falls out of balance, you simply move your hand in the same direction to recover the center of balance. A bicycle is balanced by exactly the same principle.
Edit: Relevant Video: https://www.youtube.com/watch?v=2Y4mbT3ozcA