Hacker News Reader

Comment by scarface74 | Hacker News Reader

So a cron job can give me point in time recovery?

Can a cron job give me automatic failover to another server that is in sync with the master? Can it give me autoscaling read replicas?

Who is going to get up in the middle of the night when the cron job fails?

Yes I am sure my company that has six and seven figure contracts would be just as well served self hosting MySQL on Linode.

View on HN

OTOH the last "cloud" solution I've seen was a database that allowed a team to schedule itself. It was backed on Google cloud, had autoscaling backend. It was "serverless" (app engine), and had backups etc configured.

Cost to run it for 4 years: $70000.

QPS ? 5 was the highest I ever found.

You don't need this capacity. You just don't.

But it's got to be a good business to be in ...

Instead of just assuming pricing based on Google pricing (which we weren’t talking about), you could always just find AWS pricing.

https://aws.amazon.com/rds/aurora/pricing/

Storage cost of 500Gb of data for four years - $2400

Backing up is free up to the amount of online storage.

IO request are $0.12 per million.

Transfer to/from VMs in the same availability zone is free.

A r5.large reserved instance is $6600 a year.

Of course if your needs are less, you could get much cheaper cpu and memory.

Issue is, what would the charge be for a typical application for some basic business function, say scheduling attendance.

So QPS really low, <5. But very spread out, as it's used during the workday by both team leader and team members. Every query results in a database query or maybe even 2 or 3. So every, say 20 minutes or so there's 4-5 database queries. Backend database is a few gigs, growing slowly. Let's say a schedule is ~10kb of data, and that has to be transferred on almost all queries, because that's what people are working on. That makes ~800 gig transfer per year.

This would be equivalent to something you could easily store on say a linode or digital ocean 2 machine system, for $20 month or $240/year + having backup on your local machine. This would have the advantage that you can have 10 such apps on that same hardware with no extra cost.

And if you really want to cheap out, you could easily host this with a PHP hoster for $10/year.

So how do you calculate the AWS costs here ?

If you have a really low, spread out load, you could use Aurora Serverless (Mysql/Postgres) and spend even less if latency for the first request isn’t a big deal.

And storage for AWS/RDS is .10/gb per month and that includes redundant online storage and backup.

I’m sure I can tell my CTO we should ditch our AWS infrastructure that hosts our clients who each give us six figures each year to host on shared PHP provider and Linode....

And now we have to still manage local servers at a colo for backups....

We also wouldn’t have any trouble with any compliance using a shared PHP host.

You keep mentioning six figures contracts as it's something big. Most of fortune 500 companies have their own data center and still read database manuals somehow...

No I’m mentioning our six figure customers because too often small companies are seen as being low margin B2C customers where you control for cost and you don’t have to be reliable, compliant, redundant or scalable from day one. One “customer” can equate to thousands of users.

We are a B2B company with a sales team that ensures we charge more than enough to cover our infrastructure.

Yes infrastructure cost scale with volume, but with a much lower slope.

But it’s kind of the point, we save money by not having a dedicated infrastructure team, and save time and move faster because it’s not a month long process to provision resources and we don’t request more than we need like in large corporations because the turn around time is long.

At most I send a message to my manager if it is going to cost more than I feel comfortable with and create resources using either a Python script or CloudFormation.

How long do you think it would take for me to spin up a different combination of processing servers and database resources to see which one best meets our cost/performance tradeoff on prem?

Depends on how you architecture it, theoretically spinning on prems or in baremetal or vsphere should be an ansible script with its roles and some docker file regardless.

Just for reference we ""devops"" around 2 thousand VMs and 120 baremetal servers and a little of cloud stuff through same scripts and workflows.

We don't really leverage locked in cloud things because we need the flexibility of on prems.

In my business hardware is essentially a drop in the bucket of our costs.

P.s: I totally think there are legit use cases for cloud, is just another tool you can leverage depending on the situation

Spinning up on prem means you have to already the servers in your colo ready and have to pay for spare capacity. Depending on your tolerance for latency (production vs non-production environments or asynchronous batch processing), you can operate at almost 100% capacity all of the time:

Lambda vs VMs (yes you can deploy standard web apps to Lambda using a lambda proxy)

Serverless Aurora vs Regular Aurora.

Tx instances vs regular instances

Autoscaling VMs.

Autoscaling DynomoDB

Etc.

Redundant online storage sucks, doesn't protect you against

  DELETE FROM orders WHERE $condition_from_website;

So that's just useless. You need offline backups. Is there something like, euhm, perhaps checkpoints, that you can use ?

Yes, you can schedule automatic snapshots where it will take snapshots on a schedule you choose. You get as much space for your backups as you have in your database for free. Anything above that costs more.

You also get point in time recovery with BackTrack.

https://aws.amazon.com/blogs/aws/amazon-aurora-backtrack-tur...

The redundant storage is what gives you the synchronous read replicas that all use the same storage and the capability of having autoscalinv read replicas that are already in sync.

> automatic snapshots where it will take snapshots on a schedule you choose

So... A cronjob?

Does your cron job allow recovery with a minute granularity? Is your cron job also redundant? How long would it take you to restore from backup?

You say that as if filesystems with point-in-time and/or checkpoint recovery are anything hard.

So if it “isn’t hard” then show me an implementation that does that with Mysql that can meet the same recovery time objectives as AWS with Aurora?

Actually with zfs and btrfs you can service mysql stop, put checkpoint back and bring it back up at roughly the same speed as AWS can do it.

I also have done enough recoveries to know that actually placing back a checkpoint is admitting defeat and stating that you

a) don't know what happened

b) don't have any way to fix it

And often it means

c) don't have any way to prevent it from reoccuring, potentially immediately.

It's a quick and direct action that often satisfies the call to action from above. It is exactly the wrong thing to do. It very rarely solves the issue.

Actually fixing things mostly means putting the checkpoint on another server, and going through a designed fix process.

You haven’t actually tried using ZfS with Mysql have you? Do you know the drawbacks and performance penalties for doing so?

http://download.nust.na/pub6/mysql/tech-resources/articles/m...

Have you benchmarked the time it takes to bring up a ZFS based Mysql restore?

a) don't know what happened b) don't have any way to fix it And often it means c) don't have any way to prevent it from reoccuring, potentially immediately.

Can you ensure the filesystem consistency when you restore with ZFS?

a) don't know what happened b) don't have any way to fix it And often it means c) don't have any way to prevent it from reoccuring, potentially immediately.

In the example that was given in the parent post, we know what happened:

Someone inadvertently (hopefully) did

  Delete * From table

You restore the table to the previous state and you take measures to prevent human error.

Reply on news.ycombinator.com