I colo at a local ISP. I've been with them for about a year and I'm happy. Selection was easy: I wanted a local shop and there's only one in town. I had worked with them before on other projects and figured we would get along well.

I manage the hardware myself, mostly remotely. Occasionally I'll go onsite to upgrade or repair something. I buy used servers and parts from a little over five years ago or so. A lot of folks buy new and replace every five years, so this is a sweet-spot for performance per dollar. Kinda like buying a car just off-lease.

Working a cycle behind has its own benefits. If you pick a very common model you'll get cheap and easy-to-find parts. I was able to get a full set of spares for far less than what a typical hardware maintenance contract would cost per year (and I have them on hand rather than in 2-4 hours). Drivers (especially open source drivers) will be better developed and less buggy and you can probably find someone else's notes about anything that usually goes wrong.

Of course if you need cutting-edge gear, this won't be a good option. But I don't, so I take advantage!

I think whether you'll save money depends a lot on how you do things. There are efficiencies of scale that big providers benefit from, but there are also efficiencies you can leverage if you're small and flexible (like tailoring hardware to your use-cases and not doing/supporting stuff you don't personally use).

I didn't make the move to save money, but to get more power and flexibility. So far, so good!

Good luck! If you decide to dive in, I hope you'll come back and let us know how it goes!