I have wondered about a similar thing in the context of solar panels floating on the oceans: if the weather and the waves get too rough, just go drive to -5 meters or so to avoid damage from the waves and surface after the storm is over. But of course, salt water corrosion and growth of marine life on the panels is going to be a big problem with that
In WW1, Germany flew some "height climber" zeppelins, which could reach 20,000 feet (still well within the range of stormy weather), but they could carry very little payload. They were also fragile, vulnerable to damage by quite ordinary weather. I think all other airships were limited to a few thousand feet, at least in practice.
Even if the problem was somehow solved for blimps, (perhaps with an internal "diving bladder" filled by compressing some of the carrying gas), their cross section at lower altitudes would be prohibitively big, making them effectively unmaneuverable. A very high altitude blimp would effectively be a one-way vehicle, it could raise as an uncontrolled balloon (then it might even be acceptable to not solve the slack issue at all), then operate as a dirigible at target altitude.