Hacker News Reader

Comment by kuschku | Hacker News Reader

I’ve run the numbers with Hetzner, GCP, AWS, and even more managed products such as Firebase, Heroku, etc.

Bare Metal compared to GCP or AWS will save you about 75% of your operating costs.

Bare Metal compared to Heroku or Firebase will save you around 90% of your operating costs (a large part of this being caused by bandwidth, which is massively overpriced at Google and Amazon).

With those savings you can usually serve 5 to 10 times as many customers.

That might not be worth it if you’re in SV, and pay your devs the same wages Amazon or Google do (as then you’ll just pay more than with AWS or GCP), but if you’re in places where you pay half the wages Amazon or Google do4 , you can actually save a lot with this arrangement.

View on HN

Total server spend on my last startup was like $150 a month on GCE. So saving 75% is useless if it cost me even 2h.

Fwiw I started on DO and migrated off to get better stability and block storage / proper load balancing. Which DO has since added.

You mist have fit into the free egress tier. Maybe something that could be cached by Cloudflare or similar?

The egress costs for AWS, GCE, and Azure are usually the big cost driver.

Was a b2b app with ARPU of say $100. Our total all in hardware cost for that same user was maybe $.10.

So saving on server cost literally did nothing. Our employee cost where 20x, 100x our hardware costs.

Not all startups aim for millions of free uses you will make money off of later.

Definitely. But if hardware is the only expense that scales per user – as is with most B2C apps, and even many B2B products – the difference between GCP and bare metal can make you last a lot longer on the same seed round.

(The show Silicon Valley actually showed that part quite well)

Except as I told you not always.

If we optimized and saved $100 per month on hardware at the cost of even 1 engineer hour, it would be a net loss.

I think a LOT of startups are very simple from the hardware level. You can go real far on just a few servers. And at that level, who care who manages them? Do what is easier.

I am really reluctant on setting up my own server cluster on bare-metal right now, can you recommend some specific open-source tools to manage and monitor these clusters?

Kubernetes on CoreOS is a good option. Tectonic makes the setup part relatively easy.

Ok, thanks a lot. Since we're on it, is there any open-source software available that is specifically targeted at easily setting up storage servers (maybe with an S3 like SDK)? I found myself pretty much forced to use GCS because of a lack of good alternatives.

I haven't used it personally, but Minio [0] is something I've heard mentioned more than once.

[0] https://github.com/minio/minio/blob/master/README.md

Minio is very nice, but sadly it only allows one set of credentials – I’d have preferred being able to separate the access permissions of each service, but that’s not supported.

You will likely save a lot of headache to just use AWS or GCE. Running your own kube is a multiple week project.

I was a total noob at this, and started setting up a kube cluster on 10th of May, and I managed to set up a working cluster by the middle of the week. It’s actually not as complicated as expected.

I still agree with you, because no one should take setting up and running kube lightly – it’s actually a lot of work.

This is good, it means kube is about 100x easier to set up than openstack.

On the other hand, Google will run your Kube for free up to 5 nodes.

There’s actually even a hobbyists guide for running kube yourself: https://github.com/hobby-kube/guide

And there’s a few guides for setting it up automatically in vagrant, or on aws, or on Juju-managed clouds.

Sadly, Kube is very barebones – only doing scheduling, if you add a storage provider such as portworx (only block storage) or rook (block and object storage) you also gain that, but it doesn’t do any user auth functionality.

That’s something I’m atm still searching for, something allowing me to easily register and remove users, add them to groups and do RBAC, but without using ldap as backend, and with a way to get it working with OpenID Connect and OAuth 2.0, as well as with Shibboleth, Kerberos, and an LDAP compatibility layer (but with app-specific passwords there).

You can give Proxmox a try too.

Reply on news.ycombinator.com