We established the Fed (and later, the FDIC) because people were sick and tired of bankers controlling monetary policy and wiping out their life savings. How the Fed turned into the ancap Boogeyman is the real destructive force in our society.
The Great Depression, the savings and loan crisis, and the GFC all happened after the establishment of the Federal Reserve. Sure, I guess you could claim that all of those would have been worse without the Fed, but reasonable minds can differ on that without being an "ancap".
The only thing worse than a bunch private bankers controlling monetary policy, is a central bank controlling monetary policy.