Hacker News Reader

Comment by eicnix | Hacker News Reader

First of all it is important to understand that building/running your own cloud isn't only huge investment and high risk but also comes high cost of opportunity because every cent you spend in infrastructure you hope will be cheaper is not spend to increase the real value of your product. Not making this investment until your organisation and IT are "cloud-ready" decreases this risk. I have seen companies invest more than double the cost of a cloud provider to build their own cloud without having access to the features that a public cloud provider provides.

For me the questions come down to the proportion of resources to revenue and how "cloud-ready" you are.

View on HN

Hm, but how much of those features they don't have access to would actually be used? After a few decades in the trenches, I've come to (mostly, kind of) accept that on average, most companies do not push the leading edge, and settle for relatively baseline feature sets of all of the software they use. As long as a deployment hits those feature points, they don't aggressively push for learning, testing, deploying and using new features.

This is of interest to me because no matter how I slice it, when analyzed from this "baseline feature" perspective, cloud solutions of conventional software that require deploying upon conventional platforms are expensive, and I figure I must be missing the picture. I'm much more excited about cloud solutions like AWS Lambda that de novo solutions are built around, as those do seem to have more promise delivering the much-talked-about 2-10X cost improvements compared to conventional software, but still at a high cost of vendor lock-in due to lack of cloud API standards adoption by vendors.

There were a lot of basic features missing. After 2 years of initial Openstack deployment there was no DNSaaS available and the loadbalancer solution was very unstable. This was partly caused by the organisation because nobody knew how to build a cloud system and first first few iterations of this private cloud were build like traditional bare metal infrastructure. If the investment for the private cloud had been used to adapt a public cloud provider for all non critical applications the benefit of the organisation would be greater.

There are tools like Kubernetes / Mesos that can help abstract the infrastructure layer so you can move between different providers. I have seen kubernetes federations setups between kubernetes clusters on AWS and kubernetes clusters that were hosted inhouse.

Building/running your "own" cloud doesn't require owning bare metal servers. You can rent and lease for a fraction of the price of renting cloud instances. Once you start paying up front of reserved instances, the capital invested tends to start stacking against you.

You certainly can burn through ridiculous amount of money if you insist on purchasing servers outright and pick expensiv brands.

Conversely, the "private cloud" deployment I run are mostly rented servers with month-to-month contracts. No upfront infrastructure spend - we grow them a server at a time. We also run some bare metal setups and they're generally cost-competitive with rented servers, but there it certainly depends on the size of the installation.

Interesting point. In my experience buy vs rent depends on the company size.

Which bare metal providers can you recommend? How much control do you have over your network?

I recommend OVH.com. They provide bare metal servers in a semi-automated way and have locations in US, Canada and France.

For the type of application that I run, where our demand is not very elastic, I find it very cost-effective and more flexible (as long as you're comfortable with administrating KVM and ZFS). I also like that with ZFS, I can easily sync snapshots between providers at the filesystem-level.

Thank you for the recommendation. I plan to do some testing to setup a kubernetes cluster on bare metal and will try ovh for this aswell.

Hetzner and OVH are my go-to providers if I don't have specific needs that precludes it (in Hetzners case, geography - I don't think they have data centres outside of Germany).

I don't tend to care that much what control I have over the network as most of the systems I deploy are set up to allow us to tie in compute or storage resources "anywhere", which means we put encrypted overlay networks in place for all inter-server traffic no matter what to get a consistent fabric to build on top of. It costs a bit of CPU resource, but I've yet to run into anywhere where's it's been an issue.

Reply on news.ycombinator.com