To be fair though I think that the "its only taxpayers money" attitude is disappearing. But there are still so many other inefficiencies.
As for your fix. You try firing a civil servant and making it cost less than their salary plus pension.....
I can see you've never worked in the civil service. Promotion, possibly "sideways", is by far the more likely outcome for gross negligence. Sacking people for gross negligence is hard enough even in the private sector under uk law.
HR in the civil service is sh*t scared of the unions and strikes are an option of last resort. Virtually nothing gets that far.
Again as I've commented above it would require such a sea change both in law and process - all of which the unions would fight every last step of the way - that there are undoubtedly other lines of attack.