As long as the UK government emits gilts in pounds, they will never become unable to pay the resulting obligations. So: where exactly is the problem? Where are the Emperor's clothes?
2. What makes you believe that unusually high inflation would result if the UK government reversed its austerity stance and implemented some kind of public works program?
Inflation is largely the result of the normal processes in the economy whereby the relative valuation of different goods changes, and different actors in the economy strive to increase their share of the total income. If you actually look behind the scenes how the various inflation measures are computed, you will see that the different types of goods in the baskets used usually show quite different behavior. Most of the time, a few goods and services will be decreasing in price while the majority increases in price, though at (often vastly) different speed. So what you are really seeing is changes in relative price, which reflects changes in how goods are valued. Without this process, the market could not function properly.
This process is biased towards larger nominal prices on average for a number of reasons - mostly because it is rare that somebody voluntarily decreases their profit - but there is nothing inherently bad about that. You could think of it as the Red Queen Hypothesis of economics.
It is clear that, beyond those processes, inflation can be pushed higher by excessive aggregate demand or by increasing resource prices - price increases in the last months were dominated by energy prices.
So I could rephrase my question as: What makes you believe that turning away from austerity would create excessive aggregate demand in the UK? Keep in mind that the current context is one of high unemployment and idle factories, i.e. there is more supply capacity than is currently being used.
As a final note: I think it is quite disingenuous to jump directly from "there could be higher inflation" to "1920's Germany". Those scenarios are entirely different beasts.