The reason for being in the lane is safety. In many situations, impatient people who feel they are impeded will try to squeeze past a cyclist who is not riding safely in the lane. This is among the most frightening and dangerous things that can regularly happen to any road user.
I'm not trying to chastise you. I just hope that you can imagine the situation that other travelers are in so you don't hurt someone in impatience.