In order to go higher, you must enclose a larger volume of lifting gas. Eventually, the weight of your envelope will exceed the amount of lift you're able to generate with the enclosed volume.
The Japanese currently hold the world altitude record (among all professional and amateur balloons), IIRC. Their balloon was 3.4 micrometers thick, 60,000 m^3 of helium. They reached 53 km.