Airships, particularly rigid airships (Zeppelins), are massive and it's the mass that becomes a problem. The airship is inflated to a state of neutral buoyancy so it floats and can hang in the sky (cool!) but the mass has to be dealt with when you want to start, stop, or change direction. I believe that the Hindenburg required a ground crew of ~200 to control the airship during landing and mooring. It's also a problem in flying in the vicinity of strong storms.

You also have to maintain the neutral buoyancy as you burn fuel. Zeppelins could use diesel but the preference was for 'blau gas' which had a density close to air, and didn't cause the airship to lighten as it was burned. Maintaining buoyancy also complicates using an airship as a freighter; an offsetting load is needed when making a delivery, otherwise you end up venting your lifting gas.

All that said, if I had the money I'd totally blow it building a Zeppelin-style (and size) airship.