Had this idea before. I search on this internet, if recall properly, it's because to get into orbit, you mainly need a good horizontal (parallel to the earth) speed, and the ballon only give a vertical speed.

Plus a ballon have lots of frictions and also cannot lift weight rocket.

I hope it help a bit. (If a rocket scientist could answer, that would be awesome)