Its that old trope where a powerful group lobbies something in a screeching way until discussion turns to the aspects of doing exactly that which actually affect those same powerful people and the joke is "we didnt mean like that"

If you eliminate the contractors, the work doesn't get done. So either you give up on getting the things done (which is illegal, since the things are federal law) or you nationalize the contractors (which will save some money, but not the entire current budget).

We can definitely save some money if we stop demonizing federal employees, and imagining that we can improve it by creating a new layer of middle-men by privatizing the same work. But for that matter, we could save a lot of money by taking what the government does seriously, instead of as a combination football, whipping-boy, and trough.