On a normal bicycle when you lean off the vertical axis, a compensation is made by turning the front wheel towards the "inside" of the lean thus bringing the bicycle again closer to the vertical. i.e. the bicycle (with the help of your steering input) turns the front wheel to catch your fall. If you lean without steering you will simply fall over to the side immediately! Don't try to do this by the way =)

This is the same principle of balancing an upright broom vertically, handle first, on the palm of your hand. When the broom falls out of balance, you simply move your hand in the same direction to recover the center of balance. A bicycle is balanced by exactly the same principle.

Edit: Relevant Video: https://www.youtube.com/watch?v=2Y4mbT3ozcA

Care to explain what you mean?

Riding a bicycle presupposes movement and control. To accomplish the act of riding a bicycle you must be able to steer otherwise you will just fall over to the side and you will not be riding it anymore =)