Hacker News Reader

Comment by CameronBanga | Hacker News Reader

Wish they would have given an idea of how much this helped improve their AWS bill. Hard to judge the value of this implementation without knowing the $$$ saved.

View on HN

Our S3 costs were between $400/mo. - $800/mo. After SSDB it's more stable around $200/mo, so a conservative number is $200/mo savings.

A related comment about our costs:

https://news.ycombinator.com/item?id=26740911

Out of curiosity, have you compared SSDB via MinIO (as another commenter mentions)?

Asking because we use MinIO, and for us (a small online place with a single actually active MinIO server) it's been pretty good + super stable.

Hadn't heard of MinIO. We looked into another S3 caching solution that someone linked in comments here, but it came down to we already used SSDB in production. Also, a personal preference of mine is to keep all logic in the same layer: Python.

For ex: We use SQLAlchemy as the source of truth for our relational database schema, and we use Alembic to manage database schema changes in Python. We even shard our Postgres tables in Python [1]. It fits with my preference to keep this caching logic also in Python, by reading/writing directly to the cache and S3 from the Python app and workers.

[1] https://github.com/sqlalchemy/sqlalchemy/issues/6055#issueco...

Can Minio act as a caching proxy? Or do you do something manual like publishing to minio first, and then later publishing to S3 in batch or something?

edit: Found this https://docs.min.io/docs/minio-gateway-for-s3.html

Awesome! So minio can actually act as a caching proxy for S3. Very cool.

How is $200/mo savings worth spending more than like 10 seconds of engineering time on? If you have to support the solution at all you wipe out all the savings almost instantly.

We're a one-man shop, so the cost savings made sense in the long run. I already use SSDB and Redis in production so I'm very familiar with supporting it.

From business growth perspective: if you have paying customers - then such low cost savings are counter intuitive, for a one man shop.

A one man shop with paying customers shouldn't spend time on optimization, unless it's just for fun.

If I'm a customer - I'd rather see more value added functionality, rather than cheaper data storage.

But that is my opinion, hope that your customers appreciate you spending time on this.

they're not mutually exclusive

They kind of are. A one-man shop can only do one thing at once. Either they save costs on data storage, or they spend that time doing something else.

No, pengaru's right. They're not mutually exclusive. For ex: making the site faster and more stable means I spend less time fixing and keeping things running and more time building new features.

But cost optimization doesn't make a site more stable.(Remember the context here)

In fact, adding another component to optimize the cost is adding another point of failure.

Nah. As someone who has gone from a 1 person project to ~10 people and funding, taking the time to tackle problems like this is critical - it frees up an order of magnitude of time later, helps make the product experience smoother, saves you real money that, when you're one person, really can make a difference.

Generally good thinking, but few "real world" notes

A) you first need to run the service to get any customers

B) this might take long

C) you maybe don't want / can't get VC money at this stage

D) you maybe are not the most advanced dev who can properly utilitize s3 from I/O perspective, getting you to higher costs than possible

E) there might be a time period between introducing the service and getting traction which yields enough feedback, so you can start adding more business features

F) when you are burning your own money, you are more senstive to the cost side - which is not ultimately wrong

Just my 2c

Performance is a feature. In this case, it’s the perf gained by avoiding round trips to S3.

Which could have been gained by.... hosting on AWS.

Which ends up being back to - purely cost optimization of the initial cost optimization.

It seems like the whole product is focusing on cost optimization. I suspect that the OP would be better off making money by doing cost optimizations for third parties.

You could just host in AWS EC2 pay the extra few dimes and not pay ANY S3 transfer cost.

The economics don't work out on that... we migrated away from our $3k/mo AWS bill back in the day to save $2k/mo with DO. Going back to EC2 would save $200/mo but cost $2,000/mo in compute.

Have you considered lightsail? So much cheaper than ec2 and you still get to be on aws.

You can also look at it as 50% savings, which will grow with the business.

You can look at this as time wasted on premature optimization. That time could have been spent on building valuable functionality, that would actually grow the business.

How much experience bootstrapping a product do you have? It's not always a choice between spending time optimizing vs. new features. Sometimes you have time for both. Sometimes optimization means a more stable product that frees up your time for the new features.

I started a few projects from scratch... with stricter financial restrictions as well.

Splitting up the storage and execution between cloud providers was clearly an early cost optimization.

Then this cost optimization was to fix with the result of the previous cost optimization.

Adding more code that has no clear end value to your service* never makes anything more stable. Now OP has to maintain three things, instead of two. It's a classic "look at how smart I am" overengineering.

*- OPs customers definitely give 0 f's if everything is on AWS or split between AWS and DO.

or you could look at it as a performance optimization that is actually valuable from a user's perspective.

or you could look at it as a tool to help the one-man shop make it through a tight cashflow situation that's coming down the line.

This is the problem with a lot of the dev ecosystem. Many of the tool-makers don't understand the concept of a one-man shop, so scale to/from zero is never given the first class priority it deserves

I have multiple small side projects running, and cannot afford to have an expense of more than a few dollars each per month. It’s a fun project to try and reduce costs as long as your priorities are straight.

I remember seeing your comment on that previous post. I enjoy seeing those sorts of "behind the curtain" details that break down stack & cost of applicaitons.

I'm curious if you've tested what it would cost to just host on EC2 (or something potentially even cheaper in AWS like ECS Fargate) with a savings plan. At a glance, it looks like AWS would be cheaper than DO if you can commit to 1-yr reserved instances.

That would seem like an easier (and possibly more effective) way to get around costly AWS outbound data costs compared to running a separate cache and sending data across the internet between cloud providers just to save $200/mo.

Back in the day I did use EC2, but to get the same performance (especially SSD IOPs) it cost a lot more than DigitalOcean. Back then, the monthly bill for EC2 was over $3,000/mo. Switching to DigitalOcean we got better performance for under $1k/mo.

How much time is spent updating these services and addressing CVEs and other security issues when they arise?

Reply on news.ycombinator.com