> Has anyone done this and what was your experience?
It works as expected. Big cost/time upfront to get all the equipment setup, then things mostly just worked. The primary challenge is transferability and standardization.
> How do you select a colo and what do you look for?
Bandwidth availability. Proximity to the team supporting it. Reputation among customers.
> How do you manage the hardware and how much savings in time/$ is there really?
Whatever custom stack is built in the colo must be well documented, and have solid maintenance schedules, and change control procedures. The cost savings exists, but it's never as much as you think.
It's difficult to articulate the specific drawbacks. It's more organizational than operational. Imagine that you have a server dropping at random times. The logs say something ambiguous, and now you have to become full time sysadmin to some elusive problem. No one in your organization will want to deal with these sorts of issues, nor will they want you to burn time on it. There will inevitably be other tasks higher on the list. Operational issues gradually build up like other forms of technical debt. No one receives applause for updating the server BIOS over the weekend. Operational discipline/maintenance inevitably becomes last priority.