I assume the cost of bureaucracy is what we collectively pay for in taxes. If what I pay in taxes for these public services is cheaper than purchasing these goods and services through for-profit vendors, I think I can safely say I do not get a similar cost in bureaucracy.
Additionally, even if these things were slightly more expensive than the "free market" I would be willing to pay more and help share the burden of ensuring everyone had universal access.