The land is owned by the crown. In private hands it would be used productively. It has nothing to do with "misery or impossibility."
When folks don't live somewhere, there's usually a reason, and it's usually "the land is terrible" not the government is forcing people to not use it.