Please fix the AWS free tier before somebody gets hurt
cloudirregular.substack.com
cloudirregular.substack.com
As the article points out: Every other cloud does this! They all have non-production subscription types with hard spending limits.
There's a difference between "unable" and "unwilling".
The people that refuse to understand the difference are the same people that don't understand that "unsupported" isn't synonymous with "cannot be made to function".
Don't be that person.
If you have a large account and you're in regular contact with an AWS sales representative, pressure them into making this happen. Even if you work for Megacorp with a $$$ budget, keep in mind that your future hires need to start somewhere, need to be able to learn on their own, and need to do so safely.
Don't walk down the same path as IBM's mainframes, where no student anywhere ever has been able to learn on their own, making it a dead-end for corporations who pay billions for this platform. You, or your company will eventually pay this price if AWS keeps this IBM-like behaviour up.
Think of the big picture, not just your own immediate personal situation.
Apply this pressure yourself, because the students you want to hire next year can't.
Enterprise deals can end up changing the story it seems
Not all Azure subscriptions types are like this though.
Also documented at https://docs.microsoft.com/en-us/azure/cost-management-billi...
Make a SKU that's not for production with a $100 setup fee to prevent abuse and prepaid credits for consumption.
The best you can do is build your own service to monitor your bill and manually cut off your billing account if it gets too high. Hope you did it correctly because there's no good way to test it! And the bill notifications can be delayed an arbitrary amount of time with no guarantees, and disabling billing can take an arbitrary amount of time too, so you can still be screwed even if you did it correctly.
I think it's a glaring hole in cloud platforms. I get it, it's hard to implement in a way that is forgiving and doesn't cause a bunch of support tickets from people who set their limits too low and made their site go down. But that's why people use cloud platforms, so they don't have to implement the hard part. It's beyond me why the cloud platforms don't do something better here.
You're talking a 'get paid/don't get paid' decision here.
When you look at this on a high enough distributed level, across enough users, this is the profit margin. Enough people can screw up and then be able to pay the cloud provider, that it is part of the business model. You're required to run some chance of producing a legitimate megabill.
Why be forgiving when your business model requires that you set traps for fools?
Basically you add a budget to your project, create a pub/sub channel, then make a lambda that subscribes to the channel and turns off billing when it receives an overbudget message to shut everything down.
Or you could integrate it into your project directly to handle this more gracefully.
You can even send a test message, to check that everything does indeed shut down, but the whole process is extremely clunky and error-prone.
If you have Visual Studio professional or enterprise license, then you can avail recurring credits, which are useful for learning
The possibilities are endless. I personally don't see the IBM comparison.
Same with any technology, you should read first all the documentation that there is, watch all the videos that AWS releases every year for free where they explain in detail every service that they have, and try to explain the best practices of it - this is all for free, or you could spend $1000 for a course, instead of risking to lose $10.000.
If you jump in to the water without knowing how to swim, and then get angry at the water because you drown - well...
After 18 years of age you become a grown up, because your parents are not responsible for you anymore. You become responsible for yourself. That is what distinguish a child from a grown up.
Of course a parent needs also to tech you responsibility and what it means.
Mine for example never did, and I had to learn life the hard way. At first I was blaming others, but then I realized that to really grow up, I need to stop blaming others and start owning my mistakes.
So, it is not expected for anyone to know everything, but it is expected that if you want to learn something new, you need to first research the topic.
I don’t get the point that AWS docs are free. I mean, they are free, but the services are not. Do you pay for a visit in a car dealership?
Real insurance policies have maximum coverage limits, so, while it reduces your liability by a capped amount, it doesn’t, strictly speaking, limit it; you still face unbounded potential liability beyond your insurance coverage.
on edit: this is not to say that I believe an insurance for this kind of thing would really work (although maybe that is because I know nothing about the insurance business) just that I don't think the problem with the insurance would be that you could always end up getting billed more than you were insured for.
There's a big difference between owning up to your mistakes and taking on a bunch of unnecessary risk in situations where you know it's more likely you're going to make mistakes. IE: Learning, testing, etc..
In fact taking on unnecessary, possibly unlimited financial risk in a situation like that is a mistake. Want to own that one?
It also doesn't solve the problem. If the user makes a mistake they still lose a ton of money.
It's a really interesting problem. It's getting too big and too scary for a new developer to jump in and try AWS services, which means new devs will move on to newer services with better UX. I wonder if this sort of problem is one of the few existential threats AWS could actually face. Its possible it could end up as the Oracle or Salesforce of cloud services, very successful in enterprise but definitely not a good thing to have on your resume if you want to join a startup...
I don't believe there will be any new services with better billing UX through competitive pressure. You're basically asking the competing thing to take less money in order to win, as the distinguishing factor. It doesn't make sense as a way for them to win against the larger thing taking more money.
Probably the only practical solution would be arbitrary legislation forbidding the practice, as if it was anti-usury laws. Basically taking the guise of society and saying 'you may not do this, even if people are dumb'. And then you've got a problem (at scale) of people figuring out how to do hit-n-run megabilling and walking away under the legislation, having intentionally taken what they 'mistakenly' did. An 'oops, I accidentally a bitcoin' defense.
2. If money is a concern to you, then I should focus on learning how exactly the billing works and how to monitor correctly. This way you can build a product the right way, not to mention AWS by default has limits on their services set with limits that prevent you from doing something incorrectly. For example you can only make 5000 requests a sec on the API Gateway, you can only have 1000 concurrent lambdas, you can spin only 25 ec2 instances, ecc... (true, not all services have limits like this - but then again, if you want to use one, the first thing you should do is check the pricing page, this is what I do fro every new service that I'm planing to use).
3. AWS is not for developers, it is meant for SysAdmins and DevOps (true that some marketing materials are not clear on this), they should be the one configuring it to allow developers to host their code. If you want a turn key solution, then there are better solutions, like Heroku - incredibly easy to use and understand and have a much simpler billing structure.
With AWS you can do anything you want, AWS provides lego blocks, what do you build with it is up to your imaginations, and for sure it is not meant to be use directly by developers who have no idea how networks, computers, databases, cpu, ram, policies, storage etc works. Developers should focus on coding, and SysAdmins and DevOps should focus on managing the infrastructure.
And if you want to learn AWS because you want to be a SysAdmin, then it is true, that AWS could have a plan for beginners with even smaller default limits, and limits set on everything - this way you could more safely play with what they have. This would be a nice things to have in this case for sure.
But because they don't provide such thing, you need to be the responsible one, and start learning AWS the right way, and not get in gun blazing, and expect all will be ok. My recommendation is to learn one service at the time. If you do this, over the years the acquired knowledge will be gold. Plus the more services you learn the right way the easier it gets.
> Select a learning path for step-by-step tutorials to get you up and running in less than an hour.
Do you think it's irresponsible for AWS to encourage beginners to try their service when they apparently only intend it to be used by those with a computer science degree and 5-year apprenticeship under an experienced sysadmin?
It is very dangerous. If you select the full-stack tutorial you get: "Time to Complete 30 minutes". It should say: "30 min to ruin your life" ;)
If you want to really learn AWS, then this page should be used as a reference of how to design a stack. If I were you I would read the tutorials to see which services are needed for a solution, but before doing anything, I would read the docs for each of those services to really understand them, then I would go back to the tutorial and actually do it, and - MOST IMPORTANTLY - I would read the pricing page for each service that you are going to use.
> Do you think it's irresponsible for AWS to encourage beginners to try their service when they apparently only intend it to be used by those with a computer science degree and 5-year apprenticeship under an experienced sysadmin?
100% - when I started working with AWS in 2016 I had a very hard time figuring it out, because I was looking for the simplicity the the marketing team was writing about. I really don't like what the marketing team tries to tell you, because it dose not exist.
Regarding an approach to learn about AWS, I would start with all the serverless services that they have, since the pricing for most of them is ideal for beginners (WARNING - read the pricing page for each since not all have a free staring plane, like S3 and DynamoDB) and for simple weekend projects.
For example, I did build this project a while ago: https://github.com/0x4447/0x4447_product_s3_email, if you scroll down to the pricing section you will see this:
``` All resources deployed via this stack will potentially cost you money. But you'd have to do the following for this to happen:
- Invoke Lambdas over 1,000,000 times a month - Send and receive over 1000 emails a month - Perform over 10,000 Get and Put operations and over 2000 Delete operations in your S3 Bucket - Exceed 100 build minutes on CodeBuild - $1 per active CodePipeline (must run at least once a month to be considered active)
The only payment you'll encounter from Day One is an S3 storage fee for emails and CodePipeline artifacts. ```
So you can have a stack that is actually doing something very useful that costs not even a $1 a month.
It is possible to pay $0 to AWS, but you need to first understand AWS to be able to do it, another trivial example of a tiny project that is useful and cost $0 to run: https://github.com/0x4447/0x4447_product_secure301
The last point would be: don't listen to the marketing material - they are there to sell you AWS, marketing never cares about reality.
I also recommend this website https://awsvideocatalog.com - pick a service and watch all the keynotes AWS has on that service, if you'd spend 1h a day, in 6 months you'll know more about AWS then anyone else complaining here.
Anyway, life gose on, and it was overall a good chat :)
> studying years to be a doctor [...] Should you just start cutting people
Spoken like someone who’s never used AWS or been a doctor. Your analogy is horribly, badly flawed. Doctors do start cutting people, in the US nearly all med students dissect a cadaver at the start of their first year. What you’re suggesting is doing years of pure documentation reading, unguided by other people or a curriculum, before practicing AWS, which would be silly and a waste of time. People learn by practicing, which is why med students dissect cadavers, and which is why AWS offers a “free” platform to learn by practicing, tutorials to guide the learner, and advertising to attract learners.
> If money is a concern
Maybe this is why AWS offers a “Free Tier”? https://aws.amazon.com/free/
> AWS is not for developers
That’s not what AWS says https://aws.amazon.com/developer
People in this discussion are complaining that the free tire is misleading, plus not all services are covered. And it is true that if you turn on a bunch of server and you don't pay for a year, but forget about them, you will be charged the moment the year passes. Not to mention that you will be charged if you use the CPU of the free tire server to much - which probably very few people know about.
> That’s not what AWS says https://aws.amazon.com/developer
And my point is that the marketing of AWS is misleading, they try to convince you that if you don't know anything about computers, but you know code, you will be able to manage AWS. This is very misleading because AWS tries to make you think that AWS is a service like Heroku, simple to use, and there is just one button to push to make it all work. Completely false. I've seen countless AWS accounts that were completely misconfigured by developers who thought that AWS easy to manage. A basic example is the autoscaling of EC2. People will go to the autoscaling section of EC2 "enable it" and be superseded when it dose not work. Where the reality is that you have to do 8 other things to make it work, not to mention the work that needs to be done in the OS itself.
This is simply not true, to the best of my knowledge. GCP has budget limits, which can send alerts, but outside of manually catching a pubsub and shutting down billing on your own, there are no configurable hard billing limits.
I'm unsure what to make of the rest of your rant.
We go over our budget "limits" literally every month. We do this because they are simply alerts, nothing more. We have separate alerts set up for 25, 50, 75, and 100% of our "limit" so that we can track variable usage and catch spikes early.
We, of course, pay our full bill every month - over the "limit".
https://cloud.google.com/billing/docs/how-to/notify#cap_disa...
I'm not sure if that's what you meant by "manually catching a pubsub" -- you have to manually set up the script once, but it will run automatically when triggered.
AWS offers an AWS educate account already - go here to learn more
https://aws.amazon.com/education/awseducate/aws-educate-faqs...
If you are a startup go through AWS activate - 100,000 to play with.
GCP etc al do not offer this either. This is not done crime- AWS pioneered this free tier model. I used Dell and you paid for everything there.
No, they don’t, the article is misleading if not technically wrong (since they use “and/or” to link billing caps and better whole-project deletion, the latter of which mitigates a subset of the problem but doesn’t address accidental unbounded spend from a project that you want active at some level) and the same general class of complaint is raised for the other clouds on this issue; the frequency of the complaints seems to vary (ordinally, but not linearly) with usage of the various clouds.
They don't. I've looked. They all have some ultra complex scheme for monitoring billing and programmatically shutting down services, which is a favorite recommendation by apologists, but none have a nice, solid "shut down everything" plan that's usable for learning and testing.
AWS does have billing actions which are fairly new, but they're still a bit too difficult to deal with if you also follow the advice for setting up a root account for billing plus sub-accounts / organizations for use.
Azure has been ignoring the request for over 8 years [1]. There's a warning they're moving away from User Voice to a product-by-product solution for feedback, so that request might disappear altogether. I reached out to them and politely asked for an update and they said they'd look into it, but it's been over a month and I don't think there's going to be an update without some pressure. If anyone has a good social media following and would like to see hard caps on spending in Azure, tweet @AzureSupport. Maybe if enough people do they'll actually follow up.
> PLEASE stop making the argument that it's "too hard" or even "impossible" to implement spending limits.
I agree 100% with this because all I really want is something along the lines of the "personal learning" account that's described in the article. I want a dev / testing account where I can learn and test assumptions about billing without risking a massive overage that I wasn't expecting. They don't have to get into the complexities of dealing with production accounts.
AWS, Azure, and GCP have all had close to a decade to figure it out, so I think the only way it'll happen is if we start asking legislators to regulate those providers. It would be relatively simple in my opinion. Cloud infrastructure providers must allow users to indicate a maximum amount of spending per month and may not charge more than the indicated amount.
I think big tech is proving they can't self regulate, so it's time to quit letting them.
1. https://feedback.azure.com/forums/170030-signup-and-billing/...
They just refuse to implement this system for non dev licenses.
Not a lawyer, etc, and don't even know what jurisdiction you are in or what providers you use, so this is just a general caution that your strategy may not generalize to other companies.
The problem isn't that it's too hard. It's that it isn't a business priority. As soon as it becomes prioritised, a legion of very smart AWS engineers will solve it.
"Too hard" is not an excuse. They have all the resources they need at their disposal to get the job done.
A culture they feel aligned with.
They are poor or high burn rate consumers.
They are curious.
But, it would seem, most of all they have different priorities that you.
I write that as someone who routinely dismisses their recruitment attempts.
Let these people fix climate change then.
Sarcasm aside, it must be boring working in a place where no problem is ever too hard.
that's cheaper than the pandemic. no reason it couldn't have been done already.
i no longer consider climate change a hard problem, the only hard problem is building political momentum without getting shot
Everything has trade-off's. When a responsible person says something is "too hard" it's because they've evaluated the consequences and have deemed them too costly in terms of time, resources, cost, maintenance, or strategy/lost-opportunity. Some things really are "too hard".
I agree it's not too hard technically but am surprised at your contradictory suggestion AWS would deploy "a legion of very smart engineers" to solve it if they decided to implement that ordinary feature. How smart are they?
I suspect the truth here is a bit more ugly. AWS doesn't want to do it because they see the pocket-change taken from accidental mishaps as a net positive. They don't care if it inflicts suffering on students and novices. Sure, they'll fix individual cases if they're shamed publicly or if the individual contacts them in exactly the right way, but at the end of the day, it's more money in their pocket. For every 200 dollar oopsie that gets charged back, how many go silently paid in humiliation and hardship?
What they don’t want is companies that are spending say $2-4 million per month to be able to put in a hard limit at $2.5M.
I would guess most companies spending millions per month on AWS are just guessing what next month’s bill will be, and have been conditioned to not freak out if it’s a million over, as long as that doesn’t happen every month. That conditioning has to start early.
Why not? It would make companies happy.
The reality is that, as much as many would like this feature, I'm not sure that a lot of large customers are actually asking for it. Does anyone who works at a large company know if they've asked for this feature before?
I can imagine why startups might want it, but also, AWS is extremely forgiving with startups and throws tons of money at them, so idk.
what do you think will happen to your EC2 instances if your payment method on file with Amazon fails repeatedly and for more than 30 days? Do you think they will just keep those instances intact indefinitely without payment because they promised not to 'randomly destroy things'?
(Had to resist the urge to post “Occam’s Razor” or some other incorrect thing to bait someone to correct me)
As someone who tries to always get things right, it somehow seems wrong to deliberately post wrong answers.
I know it won't go over well at Stackoverflow, which is one of the very best places to get accurate and helpful information.
Now I'm positive that he was doing the same thing as you describe.
The effect on me was a reduced interaction with that person, I'd much rather have an honest answer or a "I don't have time" answer.
This is a hard no from me.
How is AWS going to even know what to shutdown/remove? What if it's storage causing my bill to overextend?
Yeah, not just no, but hell no.
It would be nice if cloud providers were better at surfacing potentially company killing things via their dashboards and nag emails.
For example, their CDN has no "TB per month limit". If your site gets popular and is 100% cacheable, you could be getting a million dollar bill.
I hear similar horror stories about the various distributed databases in several clouds. They're all crazy expensive.
My business is looking at cloud compute at the moment and we have absolutely no idea how to do it safely. In fact sagemaker is exactly a product we have looked at and discounted since we cannot be sure we can do it safely without getting unexpected megacharges.
We had an incident recently where a cellular router was set up to send a text message on a GPIO changing. The old firmware didn’t have pull-ups on the input. We told our end customer to update the firmware. They didn’t and also didn’t hook up the input and left it floating. We got a £20,000 bill for text messages as a result and a soured relationship with the customer.
WHAT?
https://www.ibm.com/it-infrastructure/z/education/master-the...
https://sdl-hercules-390.github.io/html/
https://mainframe.informatik.uni-leipzig.de/
>There's a difference between "unable" and "unwilling".
True words, look at the Links.
Item 2 is documentation. This is good! But without access to the system to try it out, how self-teachable is it?
Item 3 is the one counter point. With a third party emulator though, how comprehensive/accurate is it? Would you hire someone to work on a mainframe who'd never used a real system before?
Compare that to e.g. Java. Sure there are plenty of college courses, some even Oracle affiliated, but I can also just download a Java compiler and start plugging away. .NET is there with Visual Studio Express. Oracle cloud, for everything else wrong with Oracle, has a free tier with no time limit. Microcontrollers for embedded development can be had for €20. You can install Linux or a Linux VM on your own computer. Even stuff that falls into the more enterprisey camp like MS SQL or Oracle DB has free personal licenses.
Attending a course on your Own.
>Item 2
No it's a MVS distribution (a full blown Operating System) with all the compilers 99% the same as a "modern" z/os
>Item 3
That's the S/390 emulator.
>Would you hire someone to work on a mainframe who'd never used a real system before?
Most Mainframe-Programmer where never near the Hardware anyway...so YES i would do that absolutely (you don't do Hardware near programming in Mainframes, an emulator is perfectly fine), same as a Dev who programs in a VirtualBox, no difference.
If you want to learn how to use it, and MUCH more:
https://www.youtube.com/channel/UCR1ajTWGiUtiAv8X-hpBY7w
>Compare that to e.g. Java.
You can do exactly the same with MVS, all the compilers (and more as z/os brings with) are in it.
I've shared this before on hn, but I was about to bite the bullet on a fire base subscription to back a web app I'm building. Google announced its cancellation the week I was going to subscribe, I was hitting daily limits in my testing with my garage code and wanted the extra buffer to speed up my progress. As soon as they announced the cancellation I ripped out fire base and have been working on a swap to hasura/postgres/auth0. I can't afford, likely multiple, hundred or thousand dollar mistakes for a hobby.
"Just code better"
I'm trying...
Maybe there are discounts or other processing that makes this hard.
Or, less charitably, this would lead to people optimising their costs a lot better and canceling unused services much sooner.
Then they need to start a team dedicated to finding a good way to let customers halt spending at a given limit with minimal impact on their operations.
They already win on UX (okay, okay, it's an opinion ffs), but unlimited liability makes a lot of people very uncomfortable. Those two actions would go a long ways towards demonstrating good faith in that area.
If it would cost too much, maybe they could present it as an easy way to cut expenses at the same time that they introduce a small price increase. This is a common and long-standing complaint/feature request.
I’ve used all three major clouds in production now and I dread using the AWS console. Or really any part of it, over Azure or GCP.
I’ve always thought of them as purely winning the “nobody got fired for…” mindshare thing despite having a thoroughly mediocre product.
I'm pretty sure something along the lines of this^ was posted on HN by a former AWS employee but I can't find it now.
Their UI toolkit for that at least is on point.
As someone that has tried and failed to get some small personal sites running on AWS a couple times, I'm going to have to tag this snippet with [citation needed].
Funnily enough, I've never ever been able to understand what Heroku is, or how to deploy anything useful on it :) But I'm old, and I've always found it easier to tinker with nginx configs.
Tinkering with nginx is good, but what if you want to add a database, a cache, a logging service? What if you want to automatically build code when you push to github? What if you want to easy clone multiple environments (for staging, prod etc) without never doing an SSH into server? What if you want to have versioned releases that you can easily rollback to a specified version or push a branch into one environment and another branch into another environment? And you get a workable subdomain for every environment you build with SSL enabled. All of that can be scaled down and up with literally few clicks.
I... just add it. How do I add it in heroku where everything is "just create an app and push it" and everything is separate "dynos" (whatever that is), each with different pricing, storage, bandwidth etc.
Thing is: if we're looking for free, that may make sense. you still need tinkering though: you need to figure out what the whole dyno thing is about and how to connect from one dyno to another, and so on.
However, if we expand this a tiny bit to cheap and/or free, then simply running the lowest-tier server at Digital Ocean will be a better proposition. And installing a database these days is basically the same thing: run a command to install, all you need is to connect to it. No tinkering. And it will still probably scale significantly better than a free tier at Heroku :)
Once again, I'm biased and old, and can afford to spend money on side projects. I know that sometimes even $15 a month is out of reach (oh, I've been there). But yeah, this sorta prevents me from ever understanding Heroku :)
I compared[1] it to a bartender who walks you through an entire sale and only at the last second rejects your purchase for being underage (instead of denying you the at the initial request for something alcoholic).
Of course, that’s one of the minor things in the grand scheme.
[1] http://blog.tyrannyofthemouse.com/2016/02/some-of-my-geeky-t...
I also remember trying to set up SFTP whenever it was first released. It was literally impossible to do what they were advertising (S3-backed SFTP with jailed homes for each user) without writing a manual json config (I never got it to work). I had built my own solution for this exact thing on EC2, using a bash script more or less, and thought the hosted option would be less of a maintenance burden. Needless to say I quickly gave up.
Why you can’t look at the load balancer, look at the listener and then show what’s in the target groups is beyond me.
CDK is another tool that build on the cli and boto3 concepts, and also manages the orchestration and dependencies.
Having to go back to the console isn't a fault of CDK. Learning the right tool to use for a situation is part of the learning curve. I go back to the console all the time to look something up quick or to understand what config options are available. Or I repeat the same steps in the console enough times that I get bored with it and automate
Edit: also I tried and have given up on CloudFormation more than once. CDK is like a wrapper layer around it, and has been pleasant to use.
I know it’s rhetorical and a lesson learned myself, but yeah… I would expect folks to learn to use this tool to help manage costs this way.
I'm not doubting that the situations you describe are true, but abandoning resources like that is an AWS-lifecycle problem, not really a Terraform one.
https://docs.aws.amazon.com/transfer/latest/userguide/scope-...
The more annoying part is that the service only supports public/private key logins. If you want user/pass you have to write a lambda. The lambda is pretty simple though, it checks the credentials (so it can hit any backend you like), and if they pass it returns a 200 with a json doc of role (which is just sftp assume role), policy (the scope down from above), and home dir.
https://aws.amazon.com/blogs/storage/enable-password-authent...
This touches on a larger issue with AWS though. It's that they are trending to leave out functionality and point to lambda as the solution. On one hand, I get it. The lambda solution is infinitely more flexible, but what if I just wanted an sftp for a couple users that uses user/pass?
To your final point, it is so much less maintenance. There is no server to manage, and since I want all the data in s3 anyway, it's already there. This solution replaced a server with chroots, EBS, scripts to move data to s3, etc...
I don’t use them for any projects mostly because I already don’t enjoy having to manage AWS at work all the time. I also don’t want to live/work in a world where AWS is my only option so I try and use smaller hosting providers and services like tarsnap.
AWS billing IS complicated but could be made so much simpler...
Because, if you are a company that uses AWS at scale (deploying literally thousands of resources at a time), you care more about meeting demand and getting resources to spec, than you do about the cost of an individual elastic IP...
The price of each individual resource isn't something that you want to see on every screen you touch. It literally clutters the console with information that you couldn't give a shit about when your company is making millions (or billions) on the services you provide. You care more about your service's reliability/scalability/uptime/etc than anything else. This is priority numero uno.
If and when cost analysis becomes priority, you look to see where you may be overprovisioning resources - hence, the billing console.
But for the larger players on AWS (the multi-billion dollar ones that AWS cares more about making happy than you or I), an extra $100k in AWS expenses in a year isn't a worry, it's a write-off.
This is exactly what we do with Vantage: http://vantage.sh/
We give point-in-time run-rates of all active resources based off of the region and resource/service configuration.
In addition we try to simplify people's understanding of where their costs are coming from. If anyone needs help with this, they can personally reach out to me at ben@vantage.sh
Starting with obnoxious API changes, followed by ToS changes. Hopefully that's as far as it has to go.
It's like hiring a plane to fly over your house repeatedly to tell you whether or not it's on fire. There has to be a better way.
I want a free tier with a lock on it. I'd very much appreciate advice for what tiers would fit my monthly use if I hit free tier caps, but if I got hit with $500 right now I'd be ruined.
I think my Google compute stuff is safe, but I really don't like having any doubt.
AWS is awesome when you have a large number of resources, that are created programmatically and reproducibly, with redundancy and duplicate environments.
The budgeting tools are really amazing at letting you categorize your costs and create appropriate alerts. The permissions system lets you define very specific roles.
Its complex. But if your system is complex, it gives you the tools to keep track of it all. If you have a <=$5000/month budget, it is probably too small to make sense in AWS. You can probably run your system on a couple servers.
A lot of hackathons, workshops and courses ask you to use AWS these days. Whether it's to run machine learning instances, win a sponsor prize or learn how to use Lambda, students are often encouraged to learn one of the major cloud providers.
Also it's a resume boost. It's another buzzword you can add to your resume.
I have a handful of AWS Lambda functions with a DynamoDB backend serving hundreds of clients, my bill for the month of April was $0.01.
No, a VM wouldn't cut it.
But you are right that there are certain slots where AWS doesn't make sense: There's one in the lower middle range where you can save a bunch of money by using a VM or two with your own DB servers. And there's the one where you're so big it might actually be worth it to implement the whole stack yourself.
What kind of thing do they serve? Somewhere I could read more about this kind of project?
https://docs.aws.amazon.com/apigateway/latest/developerguide...
Basically it takes different inputs and commands and uses different APIs to fetch data based on the input efficiently without the need for web scraping. DynamoDB is used mostly as a cache for common queries so I don't go over API quotas.
Most of the bots are made by me, but with AWS API Gateway I can easily generate API keys for anyone else and keep track of their usage.
No. I looked into AWS vs Azure vs Cloudflare vs Digital Ocean for something I'd like to build this year and (for my thing) Cloudflare (Workers) was the best balance of cost, scalability, and maintainability.
> I don't understand why so many people want to use AWS for such small budgets.
Serverless (I hate the term) makes a lot of sense for small budgets and projects. You're investing very little, so the downside of (severe) vendor lock-in is somewhat low. The upside is HA infrastructure, horizontal scalability, zero maintenance, ignorable monitoring (at least initially), consumption based costs, etc..
The biggest problem with all of them, except Digital Ocean, is that it's difficult to "own" your data in the sense that you can download a copy of a DB and keep using it somewhere else. IIRC Digital Ocean has a fairly nice managed PostgreSQL offering, but it still scales similar to a traditional DB (ie: not automatically).
The next biggest problem with AWS and Azure is that you can't figure out what anything costs, at least not easily. For example, I know for a page I'd want to serve via Cloudflare Workers, I could do 1 hit = 1 point read from Azure CosmosDB, but I couldn't figure out if the pricing includes egress. Just look at the pricing page for CosmosDB [1]. It's ridiculously complicated and that's one service on Azure.
1. https://azure.microsoft.com/en-us/pricing/details/cosmos-db/...
I closed an AWS account for this reason just a few days ago, we hadn't used it for a while but there was no clear way to remove our credit card so it felt like a risk just to have it open, what if a developer logs in to mess around and accidentally flips some switch that smacks us with a charge of a couple grand... unlikely, sure, but the fact that it's even possible is terrible system design. Better to just nuke the account and move on to other cloud providers that don't make it harder for me to sleep at night.
The Trough of Disillusionment for AWS is going to be Bastille Day levels of ugly.
I still get bills on the 3rd of each month, followed by a "we can't charge your card" (I removed it when they pulled this shit) followed by a few days later you get the "hey we're gonna suspend your account if you don't pay"
Yeah cool, I only closed the account 2 years ago now, go right ahead and suspend/close/do whatever you've gotta do, I stopped caring.
It infuriates me the hoops we need to jump through just to understand how much a single virtual machine costs. Or in cases like yours - to find out what resource a charge was for. Impossible.
Designed to be impossible. FFS. We would spend more money if we were able to manage costs better.
I think it was either related to db backup or some encryption keys but I couldnt find what and where to delete to get rid of this charge.
Nowadays I use aws only for route53 and s3.
Sure, I called customer support and they reversed the charges. But something the nice lady on the other end said as she chuckled: "This happens all the time".
DigitalOcean is the worst with the dormant accounts. Just got dinged around $2.40 on my credit card. Going into DO I could not find what was causing that charge. There was nothing there. Wuuuttttt.
Wuuuttt? Fraud is what.
Maybe it's something I missed. Maybe its some hidden feature that I did not turn off. But I deleted all my droplets, all my IP's all my firewalls, etc, etc and could not find anything else.
You should be able to look under your bill for a breakdown on what service the charge is for.
This is not a mistake any company should be able to afford to make. Ever. Under any circumstances. If it really is "just an error" they need to making amends in a very public and obvious way that is clearly expensive to them to show good faith.
But we don't get that. We have to fight from a position of being in the wrong when we have money taken from us by these fraudulent practices to overturn them.
It is necessary to go HARD at the morality of "accidental theft" because it is so prevalent and because all these companies clearly don't consider it a detriment to their reputation.
You've been diligent. When does it become their responsibility to not reach into your pocket when you clearly don't want that. Or anyone's pocket. I'd bet quite a sum that where there is one there is many.
Did you look at the PDF invoice in the Billing section? I have never seen an invoice where the line items didn't add up to the amount charged to my credit card. If there's just $2.40 on there with no explanation, I'd open a support ticket to complain.
(While looking into this, I was surprised at how minimal DO invoices are, however. For GCP, I'm used to seeing on the order of a million line items per month. Seeing only 3 on my DO invoice was surprising, and could definitely lead to a case where something isn't accounted for correctly. But, I bet support will fix it up for you.)
I've seen people mail in checks worth less than the stamp it took to send them. It's a wild world out there.
We do try to determine where the discrepancy started and watch for indicators of a bigger issue.
I have to imagine that these things are manual and a human has to do it, so they'll sort it by the size of the account and do what matters
People forget to clean them up after shutting down the instance (probably kept in case you want to keep your IP)
Considering the whole IPv4 extinction that's coming down and all.
Not saying it can't happen, but I've never seen an 'Other' listed on the bill details screen. As I said above, it's detailed to a fault.
I haven't used Lightsail, so I'm not sure if those snapshots end up in the general ec2 console. But, if you already checked the LS console and didn't see anything, then they must be in the ec2 console.
> But something the nice lady on the other end said as she chuckled: "This happens all the time".
Is better then if they said: "We reminded you about the consequences of not deleting, we have done nothing wrong."
Oh I used to get this, for me it was snapshots of a VPS I destroyed. Completely my fault, but yeah it was annoying to figure out at the time. Probably worth checking you don't have any reserved IPs, disk backups or snapshots left over
I have been spoiled by my Internet provider who year after year they will automatically increase my speed as they made it cheaper, as I paid the same amount for the service.
I didn't go bankrupt, but it proved to me how scummy AWS can be given that actually trying to use less is what got me charged.
Anyone who seriously argues this is 100% unaware of the quagmire that is AWS billing. There are companies with entire teams built around just optimizing AWS billing - it's whole unsurprising that some AWS feature actually spun up 5 separate AWS features that end up being billed.
The issue is that when students are "learning" "cloud", they are not really learning cloud as much as following tutorials to click together turnkey solutions. Dynamo DB, ElasicSearch, and whatever else are all business-oriented, quick to set up services. And such, these services are hiding what is going on underneath, with the expectation that the company will cover the costs of whatever those services run. This is a standard buisness practice.
If you want to learn how to put together cloud solutions with those technologies involved, you don't even need an AWS account. All you need is a decent laptop or a desktop, with docker or VM software, where you install everything yourself and learn how to configure it (since all of the software is free), do all the networking yourself. This is what actually learning the cloud involves and translating those skills to AWS is very easy.
Perhaps the blame is on the universities and/or tutorials that push the students towards creating free AWS accounts, but regardless, there should be some incentive to takes one profession seriously so you don't end up making this mistake in a company that will not have protections in place that people want for free tier. And money is as good as incentive as any. Id rather people be smarter rather than treat ignorance as a standard and put blame on others for not expecting people to be ignorant.
For example, in the article, the poor student was using Amazon SageMaker to likely build and train models. Blaming the student and saying "why didn't he just buy his own GPU" when Amazon offered 50 hours of free training comes across of out of touch. If the student had the money for a decent desktop he wouldn't be crying over $200.
Amazon could fix this problem easily by introducing a billing cap. While others say it's malevolence as to why Amazon doesn't build one, I think it's more likely that even Amazon couldn't even add this feature to their sprawling billing infrastructure even if they wanted dto.
This works perfectly for going to cost explorer and seeing exactly where you are spending your dollars.
I don’t think this is a very high difficulty exercise.
AWS console is some next level outdated shit that needs to be improved. GCP's console is way cleaner IMO. I really hope AWS fixed this after this happened to me, but somehow I doubt they did.
¯\_(ツ)_/¯
Reading the docs I got the feeling that the product is designed for big cooperations only. But not for someone who wants real price control. The free account just made no sense to me.
I ended up renting a cloud server ~3€ a month. (Not AWS) Looking at this thread this was the best decision I could make.
This would be good.
I don't normally do "cloud" stuff. It's just not my skill set. But I have looked at it on occasion and one thing that turns me off is my inability to know if I'm going to fuck myself with a large bill from some of these services.
I don't know how to work with AWS. I don't want to pay $$$ for a "course" for something that I can explore with a "free" tier.
I logged onto AWS. Looked around, figured I had no idea what I would be charged and what the "free" tier meant I could do exactly.
So I logged out and haven't been back.
Even for something as simple as S3, I'm hesitant to use the real service during development because it's so easy to stack up charges. For one of my accounts, the usual bill is <$10 but last month it was $27 because there was some network flakiness that resulted in excessive transfer.
More recently, I was looking for a cloud GPU provider, and tried a different provider that I won't name. I tried out a ~$3 per hour instance, and shut it down after maybe 20 min. A few hours later I (thankfully) started getting billing alerts that my bill would be $1500 ish at the end of the months if my usage kept up. I couldn't find anything still running, could not get anything from support (they had some kind of chat support that told me to open a ticket), first opened a ticket then after and then after ab hour shut down my account as a last ditch effort. At which point they promptly emailed me an invoice for the $16 I had incurred during the time before I cancelled. The help desk relied to my ticket like a week later, asking for more information.
Needless to say, I won't ever use that company again, but it could have been a lot worse, even more so if I was a student or someone who blew their whole experimentation budget on whatever mistake I made.
Hinting at the name like this seems unfair to any other companies that might reasonably be interpreted as "[Vague description of provider now removed]"... especially since I can't decide if you're sarcastic about [piece of description] part.
At the very least [two names] are big names that you could plausibly be referring too... and I could see only one name coming into many peoples minds and then them assuming you're referring to that provider.
If that's not the case in your country, ask them for a different procedure. It's not something they intend to be onerous.
That said, this sounds unreasonable and if you're not in the US this may violate consumer protection regulations if they didn't require the same level of verification to sign the contract in the first place.
Edit: as the other comment mentions, most banks where you have an account offer this for free. My local UPS store where I have a mailbox also offers the service to me for free.
(Finger print is not legally required though.)
American notary: someone who doesn’t have a history of fraud, sat through a workshop and passed a test.
German Notar: a very specialized lawyer, bills like a very specialized lawyer.
Theoretically, you can get stuff notarized at US embassies and consulates, for $50 and an appointment, must bring your own witness(es). That of course has not been an option much of the past year.
Alaska
Arkansas
Iowa
Kansas
Kentucky
Loisiana
Maine
Massachusetts
Puerto Rico
Tennessee
Vermont
However, there's a much simpler route: Amazon is a US country, so you can simply use notorize.com for $25.
In Oracle Cloud Infrastructure, after your credit is exhausted, you have to explicitly opt in to billing or else they stop all paid services for you.
Moreover, you can choose to keep billing disabled and use their free services without fear of unknown costs.
AWS have just decided to run with a policy of offering refunds when people make mistakes. Unfortunately, some people are ignorant of this, or too timid to ask for their money back.
> 2 Compute virtual machines with 1/8 OCPU and 1 GB memory each.
> 2 Block Volumes Storage, 100 GB total.
> 10 GB Object Storage.
> 10 GB Archive Storage.
> Resource Manager: managed Terraform.
Attracted a fair bit of attention in some corners of the internet for this reason. Then they canned a bunch of accounts using "always free" stuff for myself and others. Do not trust.
I guess that turns out to be good for AWS’s bottom line.
Then two months later, I still get a bill for a few €…
I give up.
I Hate Amazon
This will scan your entire account and list all of your resources - it's actually made for generating CloudFormation templates, but it's very useful for a use-case like yours.
When you stopped the instance in EC2, EB did its job and created a new instance.
You eventually figured this out and killed the EB env, and the instances stopped reappearing.
But the Elastic IP address assigned to your EB env is still on your account, and it's no longer free of charge, because you don't have a running instance.
So you will be billed about a €/mo until you delete the Elastic IP reservation.
This is ridiculously confusing, and ridiculously common.
https://aws.amazon.com/premiumsupport/knowledge-center/close...
I had to close my account in order to stop the charges. Today when I hear anyone speak of AWS free tier for a year, first thing I warn them about is to make sure they keep track of what they create so they will know what to delete, otherwise they will keep buying Starbucks every month to AWS.
I run a small server and have a few odds and ends and get emails at $5, $15, $50, $75, and $100. Haven't broken $75 limit yet...
Realtime counting would be too difficult to figure out, our brightest minds are busy figuring out engagement metrics.
It's not that real time counting is difficult, it is that the amount of compute resources and electricity that would be needed to power real time billing at AWS scale would be astronomical. There is a reason why banks and financial institutions generally do batch processing in the off peak hours when electricity is cheaper and there is less demand for the compute resources. Now imagine AWS billing, which is arguably far more difficult in scale and complexity.
I actually have no idea if billing is real-time or not? I think it's mostly batch, but the records aren't necessarily, though they may be aggregated a bit.
The general point in this thread certainly holds: our systems provide service first, bill second, and that by throwing a record over the wall to some other internal system. It's not unthinkable they could tally up your costs as you go, but the expense has fundamentally already happened, and that's the disconnect.
It would be hard to react ahead of time. Small, cheap things like "a single DynamoDB query" or "a byte of bandwidth" are often also high-volume, and you don't want to introduce billing checks to critical paths for reliability / latency / cost reasons. Expensive big-bang on/off things, probably simpler, though I can think of a few sticking points.
It would be hard to react after the fact, too. Where does a bill come from? My own team is deeply internal, several layers removed from anything you're recognize on an invoice, but we're the one generating and measuring costs. Precise attribution would be a problem in and of itself- cutting off service means traversing those layers in reverse, then figuring out what "cut off" means in our specific context. That's new systems and new code all around, repeat for all of AWS- there's a huge organizational problem here on top of the technical one.
I could see some individual teams doing something like this for just their services, but AWS-wide would be a big undertaking.
I wish we had it- I'd sleep a little better at night, myself- but from my limited perspective, it sure looks like we're fundamentally not designed for this.
That’s hardly necessary. Let’s suppose you have some service that costs 1 cent every 1,000 queries. If you’re billing it then you need to be keeping track of it and incrementing some counter somewhere. If old number mod x < new number mod x them do some check, that’s very cheap on average and doesn’t add latency if done after the fact.
PS: Phone companies can pull this stuff off successfully for millions of cellphones. If you’re arguing keeping up with AT&T is to hard, you have serious organizational problems.
As an amusing extra sticking point, your service has no idea how much it actually costs, because that's calculated by billing- the rates may vary from request to request or from customer to customer.
Without spending way too long thinking about it, the complexity in figuring out exactly when to stop is significant enough that it probably cannot practically be done in the critical path of these kinds of high-volume systems, hence the reactive approach being more plausible to me.
I don't know what kinds of problems AT&T has, but at the risk of saying dumb things about an industry I know next to nothing about, your phone is only attached to one tower at a time, and that probably helps a bit. And I'm not sure when it wouldn't be simpler and just as good for them to also react after the fact, anyway.
Not when you're being simswapped
That said, your losing the forest for the trees, the accuracy isn’t that important. You can still bill for actual usage. A 15 minute granularity is vastly better than a 30 day one. As long as you can kill processes you don’t need to check in the middle of every action. Things being asynchronous is just the cost of business at scale.
Why that would be, I leave to you.
To be fair, AT&T in particular does prepaid shutoffs on a pretty coarse granularity, I think it's like 15-minute intervals.
I know this because for a while I had to use a prepaid LTE modem as my primary internet connection. You can use as much bandwidth as you want for the remainder of the 15-minute interval in which you exceed what you've paid for -- then they shut you off.
I once managed (by accident) to get 3GB out of a 2GB plan purchase because of this.
Of course that free 1GB was only free because I consumed all of it in the 14.9 minute time period preceding NO CARRIER.
But there isn’t a lot of middle ground between distributed, strongly-consistent credit limit checks every API call and billing increment (which is, IIRC, instance-seconds or smaller for some time-based services) and a hard billing cap that is actually usable on a system structured like AWS. Partial solutions reduce the risk but don’t eliminate the problem, and at AWS scale reducing the risk means you still have significant numbers of people reliant on the “call customer service” mitigation, and how much spending and system compromise to narrow this billing issue is worthwhile if you are still in that position?
You don't have to bill in real time.
You just have to provision funding for every resource except network bandwidth.
Customer sets a monthly spend limit. Every time they start up an instance, create a volume, allocate an IP, or do anything else that costs money, you subtract the monthly cost of that new thing from their spend limit. If the spend limit would go negative, you refuse to create the new resource.
If the spend limit is still positive, the remaining amount is divided by the number of seconds remaining in the month times the bandwidth cost. The result becomes that customer's network throughput limit. Update that setting in your routers (qdisc in Linux) as part of the API call that allocated a resource. If you claim your routers don't have a limit like this I call shenanighans.
This should work perfectly for one region.
There's probably a way to generalize it to multiple regions, but I'm sure most small/medium customers would be happy enough to have a budget for each region. They'd probably set most regions' budget to zero and just worry about one or two.
The web UI probably would need to be updated to show the customer "here is what your bandwidth limit for the rest of the month will be if you proceed; are you sure?". JSON APIs can return this value when invoked in dry-run mode.
Eh. AWS's edge network is highly distributed. Unless you want an even split of your rate limit across every possible way out of the network, you'd be much better off settling for an even split across your EC2 instances, and there's no room for bursting in this model. Enforcing per-instance limits (on any dimension) sounds pretty feasible, though.
This wouldn't generalize straightforwardly to services that don't have obvious choke points that can impose this sort of throttling, such as, I think, DynamoDB.
AWS systems are highly distributed; this kind of sharp billing cap would necessarily introduce a new strong consistency requirement across multiple services, many of which aren’t even strongly consistent considered one at a time (and that’s often true even if you limit to a single region.)
> Every time they start up an instance, create a volume, allocate an IP, or do anything else that costs money, you subtract the monthly cost of that new thing from their spend limit
For the motivating use case (avoiding a bill on the scale of even $200—possibly even $1—from a free-tier-eligible account), using monthly chunks doesn’t work; you suddenly couldn’t spin up a second simultaneous EC2 instance of ant kind after an initial t3.micro instance, which would cutoff many normal free tier usage patterns.
I mean, that’s a good way of capping if you are using AWS as a super overpriced steady-state VPS, but that’s not really the usage pattern that causes the risks that the cap idea is intended to protect against.
This is a particularly poor solution to completely the wrong problem.
Hogwash, I tried to spin up 100 of your F1 instances in us-east-1 a week or two after they first became available, and found out about this thing called "limits".
Wherever you're enforcing the limit on max number of instances per region is already a synchronization point of exactly the sort needed here.
I'm sorry, this just doesn't pass the bullshit test. Resource allocation API calls are not even remotely close to lightning-quick. There is no fundamental immutable constraint here.
> For the motivating use case (avoiding a bill on the scale of even $200—possibly even $1—from a free-tier-eligible account),
Avoiding a $1 bill is definitely not the motivating use case.
A lot of people would be happy to have a mechanism that could prevent them from being billed 5x their expected expenditure (i.e. they set their budget limit to 5x what they intend to spend). It doesn't matter that that isn't perfect. It is massively better than what you're offering right now.
I don’t have any F1 instances. Have you mistaken me for an AWS employee rather than a user?
> in us-east-1 a week or two after they first became available, and found out about this thing called "limits".
Yes, individual services, especially in individual regions, and especially a single type of resource within a service within a region like, say, instances in EC2, are often at least enough like centralized to impose hard limits reasonably well.
Billing accounts (and individual real projects which—and this is one disadvantage AWS has vs, say, GCP—AWS has only the weakest concept of) tend to span multiple resource types in each of multiple services, and sometimes multiple regions.
> Resource allocation API calls are not even remotely close to lightning-quick.
Resource allocation API calls that have high latency aren’t the only API calls that cost money and would need coordination. Heck, API calls aren’t the only thing that costs money.
The provider. What they do wouldn't be accepted in any other industry. Imagine hiring an appliance repair shop who sends a repair person that can fix your stuff immediately, but can't tell you what it's going to cost until 3 days after the work is done.
Then you get a huge bill because you wanted "appliance repair" (one of them), but ended up with "appliance maintenance" (all of them).
I agree that it seems like a solvable problem, but it doesn't make them money so why should they care?
Customer service can correct “we screwed up and have a giant bill” easier than “we screwed up and lost all our data”.
So its not going to happen first party.
(It’s also not really possible DIY as a hard certain cutoff at or before a limit, only at an indefinite interval in time and money beyond a limit. So you still have potentially unbounded expense.)
Example: let’s assume you’ve set the cut-off budget too strict, spun off another shard for your stateful service (DB for example), it received and stored some data during the short window before some other service brought whole account over budget (i.e. paid egress crossed the threshold).
To bring VM and EBS charges to 0 (to implement your idea of ‘shut down everything’) AWS will have to delete production data.
While it may be OK when you’re experimenting, it’s really hard to tell in automated way.
So, to fully comply w/o risking losing customer data, AWS will have to stop charging for not running instances and inactive EBS volumes which most definitely bring on many kinds of abuse.
—
There may be some other way to do this, maybe mark some services expendable or not, so they are stopped in the event of overspend.
What I and I expect most people want is a cap which then spins down services when they reach that cap. Nobody is going to care if the cap is set to $1000 and the final bill is $1,020. The problem being solved for is not wanting to have to ever worry about missing an alert and waking up to a bill that is a factor or two beyond expectations. I can afford my bill being 10% or even 40% above my expectation. I can't afford my bill being 500% off.
The solution seems to be to implement ‘emergency stop’ when whole account is put to pause but no state is purged. And then you’ll probably have a week or two to react and decide if you want larger bill, salvage the data or just terminate it all.
Still, it's not hard to imagine some simple refinements that might be practical. For example, an option to preserve data in DB or other storage services but stop all other activity might be a useful safety net for a lot of AWS customers. It wouldn't cut costs to 0, but that's not necessarily the optimal outcome for customers running in production anyway. It would probably mitigate the greatest risks of runaway costs, though.
Another option would be to "reserve" them in the budget. That is, when for instance you create a 10 GB EBS volume, count it in the budget as if it will be kept for the whole month (and all months after that). Yes, that would mean an EBS volume costing $X per day would count in the budget as 31*$X, but it would prevent both going over the limit and having to lose data.
We have a batch process that uses S3 as a way to temporarily store some big datasets while another streams and processes then and, once complete, removes them. This takes like an hour.
So our S3 bill could be, let’s say, $10/mo. If you went and looked at your S3 bill and saw that and thought setting a $20 cap would give you reasonable headroom you’d be sorely surprised when S3 stopped accepting writes or other services started getting shut down or failing next time the batch job triggered.
Under this system, the budget and actual bill need to be off by a factor of more than 10 ($240). And this also doesn’t stop your bill being off by a factor of 10 from what you “wanted” to set your limit to. More than the $200 under discussion.
Azure MongoDB billing was insanity. I was up to $800 to host a couple of GB that wasn't doing anything.
I'm still not sure what happened, even their support kept saying "it's priced by request units" and I kept saying "How does a handful of queries a day translate to $40 in request units?"
A year later, I think that I had a lot of collections and they seem to charge per-collection, but I'm still not even sure. Thank goodness we moved off of it after only a month or two.
GCP at least has some provision to get very detailed information about usage (but not cost) that updates in less than an hour. That, to me, is the tool for building something like "shut down our account if usage is too high". It is annoying that you have to code this yourself, but ultimately, it kind of makes sense to me. Cloud providers exist to rent you some hardware (often with "value-add" software); it's the developer and operator's responsibility to account for every machine they request, every byte they send to the network, every query they make to the database, etc. and to have a good reason for that. To some extent, if you don't know where you're sending bytes, or what queries you're making, how do you know if your product is working? How do you know that you're not hacked? Reliability and cost go hand in hand here -- if you're monitoring what you need to assure reliability, costs probably aren't confusingly accumulating out of control.
> I think the big problem is that usage collection is a few days out of date, at least for GCP. Autoscaling can react in seconds to increased load, but it takes about 3 days before that shows up on your cost reports.
That does not sound like a good reason, but more like a crappy implementation of usage collection.
I don’t see why a bunch of Google engineers can’t implement real-time billing properly, and see no reason to defend their inability to do their job.
It's far easier to negotiate dollar amounts than lost data or service uptime.
All of them have billing APIs which should, in principal, allow you to build “Shutdown everything at some indefinite time interval (and additional spend) after I exceed $X”, though you’ll need to do nontrivial custom coding to tie together the various pieces to do it, and actually stopping all spend is likely going to mean nuking storage, not just halting “active” services.
None of them provide anyway to more than probabilistically do “Shut everything down before I exceed, and in a way which prevents exceeding, $X.”
Let me guess...hosted on the cloud we're trying to shut down?
It needs to consume APIs from that cloud, but there is no reason it would need to be hosted there.
Then just given people a certain grace period to reactivate or get their data out before it's removed.
It wouldn't fix production deployments where you want alarms, not a shutdown, when you hit spending caps, but it would help people on the dev stage to avoid issues like this.
People bring this every time. You give a grace period and then yes - assuming the user opted into a "turn everything off".
Its not some impossible engineering feat & clouds are full of "out of money data gone" stuff already. e.g.
>If a paid subscription ends or is terminated, Microsoft retains customer data stored in Microsoft 365 in a limited-function account for 90 days to enable the subscriber to extract the data.
It makes way more sense for me to build my stuff using a dev/testing account. After that I'll have a good enough understanding of the resources I'm using that it's practical for me to configure more complicated cost controls using a production account.
It's not all or none.
Option 1:
When credit balance reaches a certain level (or monthly spend reaches a certain level), initiate a resource stop on every resource that can be stopped without data loss. This would still incur charges for some things like EBS volumes, S3 data, etc, but at least it would slow the bleeding.
Option 2:
I don't care about data loss, just terminate everything when I hit the threshold. This should require a double-opt in and maybe a warning banner in the console UI.
So option 2 by default with option 1 customisable from then on.
It turns out there is a non-widely-known program called AWS Educate Starter Accounts [0], which give no-credit-card access to a limited but useful subset of AWS services. The problem is that you can only get access to these accounts through student affiliation with a participating educational institution like a high school or university.
It might be more feasible to expand this program, say to any applicant who demonstrates some reasonable threshold of non-bot-ness, than to re-engineer the normal free tier.
To any AWS people reading this: I believe this could be a useful step toward solving the free tier problem, and would be happy to be a sounding board.
[0] https://aws.amazon.com/education/awseducate/aws-educate-faqs...
Sounds like a new dystopian future that is best not to be part of.
When IBM ruled or Microsoft or AOL things you had one main evil corp. We're in a period where Google, Apple, Amazon, Microsoft, Facebook can appear the hero or villian depending on the day but the sum of the faangs is much greater than the worst evils of the original megacorps. Could you envision forced unlimited billing on a free tier with no ability to limit charges on the account?
These threads always open a discussion about how hard limits would be unacceptable to some businesses, but the opposite also applies in other scenarios.
Suddenly I started getting hit with these real small charges each week. I never did figure it out and I certainly didn’t willfully authorize them. I just paid and then I annoyed them to shut off the account. I never could figure out what they were for.
I know many of you would hate this but I would love an option to shutdown everything until the bill is paid option. I bet the description of what I was paying for would have been a lot more helpful if they were losing the business then when they just auto charged!
I couldn't recover access to the account. They couldn't figure how to give me access. They wouldn't just remove my credit card from the account.
One month the charge became $10 so I called my bank and had them block AWS charges. It seemed the only way to deal with the fear that someday that would creep up and up.
[0] https://news.ycombinator.com/item?id=3356987
That sounds a lot like what I want a free tier to be. Let me play for a bit, set hard guard rails so I can't accidently spend $100, or much much worse. I mean, I can understand requiring a CC to do this, but that's mostly so I don't spin up a ton of free accounts and also so the friction to going to a paid plan is lower.
It's too late for me, my personal AWS account is well outside of the free tier, but the first time I spun on a large instance to test it out I was so insanely nervous I'd screw up and end up with a huge bill. I can totally see someone else backing out at that point out of fear, but if I knew I'd hit the guardrail before spending money, even if it meant losing an instance, I'd have totally be happier to play around.
The entire system is completely sinister. The fact that keys pertaining to S3 has anything do with being able to start hundreds of VM's in different parts of the AWS system or do whatever is bad.
I've seen companies be ruined by this, and it's in no way obvious how stupid their system is. You have to read huge manuals to know how to "only give access to s3" through a key.
Instead of starting with "no access" then adding atomized access you have to understand this extremely complex "json privilege system". Instead of just programming, this is the only allowed IP, the is the only allowed bucket, this is the only allowed service, and my max is 200usd, or something to that effect.
Also the fact that a key can start new services that are billable is almost criminal in my mind when people don't even gen an email when it happens - makes zero sense.
I have to disagree with this. It really is pretty straight forward.[0]
[0] - https://objectivefs.com/howto/how-to-restrict-s3-bucket-poli...
The way you do this at the moment genuinely involves copying and pasting JSON policy documents around! It's horrific.
I want this for myself, but more importantly I want it for users of software that I write. I would love to be able to build something that stores a user's data in an S3 bucket that they own (and are billed for directly) - but it's currently just too difficult to talk them through setting up the bucket and creating the right credentials for it.
What would that look like if it’s not going to be a series of access permissions and filters represented as JSON?
Security is never a simple checkbox and complaints like this about it needing to be simpler need to be backed up with an alternative. I genuinely wonder what alternative there is to the current permissions model.
It’s incredibly expressive and doesn’t take that long to understand. People who cannot master it would likely leave some other side door open anyway.
> ... and my max is 200usd, or something to that effect.
This has been a valid complaint for years. Though to solve it you need to answer what happens to legit resources when your billing cap is reached. Do all your ephemeral serves turn off? Do your EBs volumes all her deleted? Do your S3 objects all disappear?
Way to complex only needing to give permission to a single "thing". And much of the naming makes little sense to new yes.
DigitalOceans interface is easier for 90% of people.
But to be honest it's years ago that i worked with AWS, but i remember everything being way more complex than it had to be and i had to use days undestanding their enourmous interface and "usersystem". And as long as this is a problem that pops up again and again, something is very wrong with their system i would say.
If not creating a simpler interface / user system for regular users then at least give some huge warnings that unless you know what you are doing a key with the wrong JSON means access to all funds on the credit card.
Is it ironic that Amazon's mantra is to be "Customer obsessed" yet AWS is so magnificently confusing for anyone not doing it full-time?
As a designer I've used plenty of Digital Ocean, Vercel, Cloudflare Workers and other static hosts without a problem... I've never been able to figure out how to even start on AWS, and all these horror stories constantly make sure I stay away
Like the number of times I find myself in some random service/feature/part of AWS like "do we use one of these? how would I even tell?".
If I were ever going to do a personal project that wasn’t appropriate for serverless, I would just use LightSail.
I rented a small cloud instance from HETZNER for ~ 3€/month and host everything myself. If I need a DBaaS I go with the MongoDB Atlas or ElephantSQL free tier.
I prefer constantly paying 3€, over a free account that breaks my neck if I miss something.
However, my application instance somehow kept running ( was a total noob to AWS ) and I got charged over $300 the next month when I got a monthly report in an email. I panicked and literally just deleted my account. Yes, I just nuked my account like the article mentions. AWS never reached out after that.
Businesses, the entities that are paying the most money to AWS, will NOT want a hard shutdown. When you generate revenue off of your SaaS service maybe you'll understand.
No, I won't be pushing my TAM to enable a forced shutdown due to budget metrics.
Not to mention, how does AWS decide what to shutdown and what to delete? It's not like it's only running resources that cost money, what about all my data that's stored???
I don't think anyone is arguing that any account MUST take advantage of hard shutdowns. Only that it should be an option for those who truly do want it.
> How does AWS decide what to shutdown and what to delete?
That's a much more interesting question. It tends to be services, rather than storage, that are the root cause of these outlandishly large surprise bills. But if, by chance, it's S3 that's running up the bill, what would a hard shutdown mean? Which data does Amazon delete first to get the account down to its hard limit?
went through the account deletion, and got yet charge/another bill today - account is deleted, so can't log in to see why I'm being charged.
hopefully their support will help out, but not holding my breath.
editing to add: billing was for db backups, I terminated the db, and no clue how to remove the automatic backups it made. of course I can't log in to look any further.
A $200 charge would have been devastating to me when I was in college. As in, I may have to beg for food devastating.
You can spend $500,000 on AWS in 24 hours, it's not difficult, it's not easy but you totally can blow past your "budget" because it's not a hard cap.
Azure comes closest in the dashboard but still misses some items. GCP has an Asset Inventory page in the Security section for an organization. AWS can use the Tag Editor to browse all the regions. It's notable that none of them have a single clear page though.
AWS pretty clearly isn’t raking in money on this. Even at your imaginary 2 million a year, that’s a drop in the bucket for AWS, let alone Amazon.
I'm sure some crusader within AWS could get it done if they tried hard enough and collected the data to show the negative side effects and how they effect AWS.
An article like this one is hopefully going to get someone within AWS to do just that.
My gut is that this is actually the cause of most of the “evil” things that “large” companies do, where eventually they get so big and/or complicated that organizing an organic positive change isn’t plausible.
I have a friend who I know didnt get a refund.
There is someone elsewhere in this post who didnt get a refund.
I don't know what the refund rate is, but it isn't 100%.
Obfuscated billing helps Amazon with $350/hour engineers, where an engineer figuring out where $10k is going each year isn't a top priority. Been there, done that.
Obfuscated billing is definitely not there to hit free tier or poor folks. AWS has an awesome reputation for just about everything, and it isn't worth the reputation hit. Free tier is there so that:
* the student in college will pick AWS in their first job
* the random engineer will prototype something on the weekend, and Amazon gets millions of business
* the random pre-funding startup starts building on AWS, and if it goes big, so does the account
And it works. Amazon has made millions based on things I've built on free tier. AWS' problem is that my ONE YEAR free tier ran out probably around a decade ago, and I've long since moved on from what I was doing then. If AWS were to continue to provide me a free tier, the same thing would have happened a few times since.
Anything that they can get from "naive students" and developers who don't notice small recurring charges is so utterly insignificant that it can't justify any decision whatsoever. If the public relations aspects of it cause even a 0.01% change in AWS growth, that's already $5 million of lost revenue; if you can assure developers worldwide that it's not so risky to try and adopt AWS a bit more and get a 1% extra growth, that would be worth $500 million and justify walking away from all kinds of irrelevantly small cash flows.
When you stay in a hotel room longer than you're supposed to and they charge you for another half day, or you eat food from the mini fridge, or buy a pay-per-view movie, etc, you could claim ignorance, and create stories about how terrible it is that people get charged for these things unknowingly. But Holiday Inn's bookings are still not going to take a noticeable hit.
Moreover, they don't even need to offer a Free Tier to get people to use AWS. They're AWS. They're the gold standard. It's like saying IBM would need a Free Tier (back when IBM wasn't trash). It wasn't a question of whether you should use IBM, it was whether you could afford it.
I ran to my computer too fast to have time for a panic attack. Turns out my company bought some services through AWS and it was all kosher.
I wish it wasn't possible to rack up such a bill by accident so I don't have to worry.
Also, the cloud is someone else's computer. It is either AWS or GCP or something else. Move to what works, for you.
We tried to exhaust the supply—one day the credits vanished. I was getting married that month, and it took that long to accrue a life-changing amount of real dollar debt. I sent a quick email asking DO to extend the credits. The last email I got from them was that "we expect you pay the full amount".
What I'm going to say next should not be taken to detract from that in any way.
(And I know you don't want to hear it, and I could just save everyone some trouble and not say it. But I'm going to say it.)
When I read this:
> please help I made a ticket and called support but i really need to make sure this is dead please Im 20 i really dont have $200 for them please help
I honestly did a double take when I got to the "20". If you'd put a blank in there and asked me to fill it, I would have said "13". I'm still not 100% sure that the blogger didn't alter the age to obfuscate personal details.
I don't mean to single out this person as especially immature, quite the opposite: the interesting thing is that they assume that a 20 year old is obviously a smol bean who could not possibly be expected to figure out AWS billing or come up with $200, _and we agree_. The notion didn't stand out as surprising to the blogger, or to any of the other commenters on HN so far.
I'm not saying they're wrong! I just find this a remarkable signal of how far we, as a culture, have gone in extending childhood well into college age.
AWS is not cheap. A lot of people seem get bitten by this.
> I’ve personally got a dormant AWS account that’s charging me cents every month, and I bet you do too.
Hummm no? And if I have a charge line and I can try to chase it. It's not like that cent is nameless, it has a name and it has a way of figuring it out?
I mean, it can be not so obvious but it's not like it's totally opaque neither
After the 2nd month I get a huge multi hundred dollar bill. They claimed over the the 2 months I had used 2k-3k hours of time an their free months were limited to 750 hours a month of usage.
As I pointed out, that should be enough, as 24*31 = 744, so it should be impossible for me to use more than 750 hours a month.
They claimed that I must have been dialing in from multiple locations. I denied that and said they should have records of where I was calling in from. It took weeks for them to "forgive" the debt, without acknowledging that their billing was broken.
I always wondered how many other users got hurt because of this (my guess is not that many, as relatively few people were keeping their connection alive 24/7 via compuserve back in the 90s)
I don’t understand why anyone would use AWS when there is a risk of being charged stupid amounts of money if you screw something up. Are there advantages to AWS that I’m not seeing?
My current workaround is finding some SaaS provider that is willing to take the risk on free accounts and act as a middleman.
Currently using aiven.io - downside: this covers only a small subset of services that aws offers.
"ALERT-1: Session time behavior and instance types in your Starter Account will be changing on May 11th, 2021. After this date: 1. When your session ends, your resources will be “stopped.” You will be required to re-start your resources when you start a new session. 2. Updates will be made to available instance types. We recommend you to complete currently running work in your Starter Account by May 10th, 2021 as work using instance types that are no longer supported will be lost after that date."
Im still thinking about what can I do with 3 hours (duration of a session) of EC2 computing power...
I don't know if this is a mistake on their part, but I haven't been charged in the last 6 years or gotten any emails before that. But it's still worrying because the account is closed and I have no way through standard support to know why they think my account was using their services
We had promises of this as ecommerce took off, to help avoid fraud. But it seems these don't get widespread adoption, probably because purchases are easy to get chargebacks for (in general).
A hard limit for cloud purchases is surely needed. But because the cloud providers aren't giving us this, I'm wondering if a solution like disposable cards could?
2. Your payment method refusing to accept charges doesn't mean you're still not on the hook for them. Technically they can still send your bill to collections and wreck your credit report. They probably don't do this, but it's still a bad bandaid solution
1. Not sure, but I think that's the problem I'm asking about. Because prepaid or auto-generated cards are not mainstream, big vendors don't need to accept them.
2. Agreed, but at least this puts one more layer of control back to the owner. AWS doesn't have my birthdate or social security info (US), so it would be harder (not impossible) for them to "wreck your credit report". At least this approach puts an extra barrier in the equation.
We send out regular cost report emails to try and help folks avoid situations like this and have some future plans around anomaly detection to try and help out in advance of things like this happening.
I strongly encourage folks to sign up and let us monitor things on your behalf. We have a free tier that likely covers a lot of personal users here.
For example.
Starter ($5/mo) - Use up to 100,000 API calls each month.
Growth ($30/mo) - Use up to 1,000,000 API calls each month.
Enterprise ($149/mo) - Use up to 10,000,000 API calls each month. After that, $10 for every 1 million calls.
Whereas if they allowed customers to load on an amount, even a direct debit, their card processing fees would go down and customers would just get cut off when they run out of credit.
I suppose a counter argument to this is that it's hard for AWS to keep a constant eye on how much a service is costing. In which case we're back to the same argument as spending limits.
I've set a budget and an alert for the budget. Is that enough?
I can see the budget being overdrawn for the time it takes me to react to the alert but I see no way to actually shut down servies.
It would probably be possible to script by monitoring the API and then issuing aws commands to shut things down. But it would be a huge project, someone should get on that.
I have had issues with students doing the same thing, and I was being charged for dormant projects that I thought I'd deleted. AWS is a nightmare of an ecosystem to navigate.
The AWS Educate Starter Account is almost useless - especially if the student has to submit their project for external review - because it doesn't allow publicly accessible S3 buckets and has rotating credentials.
Honestly, this is needed for corporate accounts as well. Not all companies are FAANG scale behemoths who can shrug off an unexpected charge.
For a scrappy startup in India, an unexpected $5000 bill would be an existential threat.
It's so absurdly easy to run up massive bills, and there's few controls, and practically useless alerting - so you can't rely on that.
Someone spun up a 3 node m5.xlarge RDS Cluster with SQL Server Standard? I hope you notice quickly, 'cause that's going to be >$5k/month.
They don't show a minimum monthly cost for any of these things, either.
No, you're assumed to be okay with unlimited charges, and are left to your own devices to figure it out.
Thankfully, the company reached profitability so we pay them rather hefty bills now. But I guess I agree with you in that that's not a typical startup's trajectory, and their ROI from the aforementioned outreach is likely relatively low on average so am wondering now why they went out of the way to help us.
The only caveat being you have to migrate everything to new servers, which can get messy if you don't practice how to do that efficiently.
As for whether Amazon will listen, Upton Sinclair blah blah not understanding it.
Paid (but cheap): Linode, DigitalOcean
—- Machine Learning:
Free: Google CoLab, Paperspace Gradient
Paid (but [relatively] cheap): JarvisLabs, DataCrunch
—- Databases and user auth:
Free: Firebase spark
Paid (but cheap, remember to set billing alerts!): Firebase blaze
So many comments about charges that show up despite resources removed or even deleted accounts, tells me that AWS can not be trusted to present me the full picture of costs.
I was impressed, but still have been really trying to keep off AWS just to not have a mistaken repeat.
Not-actually-Free Tier
Not-actually-a-Tier Tier
Because that would imply it's not working the way Amazon wants
And Amazon is not stupid
That quote is what I think about when I see cloud hosting pricing models. Rulan was the late co-founder of Clark's Pine Factory in Northern Utah and my first boss in the 1990's.
This doesn't only hit budgetting... trying to get all the costs to go the proper billing tags (and verify that you found them all) also requires test cycles of up to 2 days.
Yes you can probably dodge that by going with a SMS enabled VoIP provider.
I would happily put a $20/mo limit on many of my accounts. They should never bill more than few dollars a month.
Something that can be resolved via getting in contact with AWS isn't one of them. I've gotten charges refunded before.
You can choose to include or exclude refunds and credits. If you exclude them, I am guessing but it should show your true cost without the free tier pricing. If they are included it would tell you the impact right after any free tier offers expired.
Perhaps setting a budget like this should be part of any organization's new account setup process.
After seeing how recklessly people use AWS when it's not their own wallet on the line, I'll wager to say that this 20-year old learned a valuablr lesson.
Why would a service allow to have eks without nodes? Will call support to get a refund!
What? I use AWS Free Tier a lot and it is the best thing for small businesses and startups.
I've personally found that it can be hard to get the approval to type in your company credit card until you've done enough research to prove that something is going to be worth the money. That leads to a chicken and the egg problem; people won't be confident until they've paid, but they can't gain that confidence until they pay! So you have to get the small "testing" use case perfect, or you'll never have any real customers. (My corollary to this is because AWS already has your company credit card, and you already have "root" or similar because of your role, it sure is easy to build whatever you want there. If AWS provides some service, you can start using it and the cost will be lost in the noise. But if it's not on AWS, then you'll have to produce some justification to use the service before you can start paying for it. I ran into that a lot at my last job; I wanted to buy $5/month services like Sentry but was told no, whereas other people could just create a m4.4xlarge RDS instance for $1000/month and nobody even noticed or cared. People really like being approvers at the time of entering the credit card, rather than at the time of actual cost accrual, and cloud providers really facilitate that. Not sure that's really helping those approvers -- it almost feels a little bit like embezzlement.)
Anyway, ranting aside, here's what we do for the cloud service I work on:
1) We have a free tier. It's really free; you don't even get to enter a credit card. Sign in and start doing your work. (We delete your stuff after 4 hours, though.)
2) For the paid tier, all costs are pay as you go. The instant you click "delete workspace", no more costs accrue. Merely having an account open doesn't accrue any costs for you, and there is no way to create a phantom resource that you can't see in the UI and delete. If you delete everything, billing is over.
One weakness that I'd like to fix is the latency between resource use and when we tell you about it. That takes a few days, so if you are playing with aggressive autoscaling, you don't really know what it's going to cost until your experiment is over. I'd like to collect real-time usage data and just bill based off of that, so that the UI can update you within seconds of your job starting. If it's too much, you can just pull the plug and not be surprised.
The next step is letting people pre-pay, and do what the vast majority of comments on this thread want: kill the compute resources when the budget is exceeded. My thought is that it's hard to ask your customers for money upfront, which is probably where the post-pay model originated. I personally always have reservations about buying 3 year reserved instances from cloud providers, even if it saves a ton of money. "What if we stop using it tomorrow!?" But there is probably a good compromise here: type into the UI what you'd like to pay for autoscaling per month, and once that budget it exceeded, run at the bare minimum "keep the lights on" level. More difficult than the alternative, but certainly possible. And very good for users -- no total outage, no unexpected bill they have no hope of being able to pay. Things are just slower for a while.
Anyway, I don't know what the perfect formula for cloud pricing is -- but it's clear that what AWS has is not quite right, and that we can probably do better. To paraphrase Jeff Bezos ("your margin is my opportunity"), what AWS has consistently done wrong for years is your chance to make it better and get paid for doing so.
Reality - very few folks want a hard billing limit.
To stop charging you AWS would need to delete all your EBS and S3 volumes, stop all EC2 instances, release all public IP's, delete all AWS directories and the list goes on. The idea that AWS would build this giant data loss footgun into their system is ridiculous.
Somewhere in AWS someone said, how could this blow up, and they came up with 100 ways, including misconfigured cost accounting etc.
That said, the GCP project based model makes more sense to me, give you more control etc.
That said, if there is such demand for hard billing limit playgrounds (I'm sure there is but not by folks giving AWS a lot of money), someone should be able to do a hosted solution for AWS that bills into their corp account and gives you a playground for learning (with a real hard billing limit). That type of approach is used in a lot of other contexts already.
I'm sure those people, given the appropriate motivation and opportunity, could provide a solution (or several) that avoids foot-gun data loss at every turn. There's all sort of solutions they could go with.
As an idea, perhaps some kind of functionality within the billing side of things:
They could, perhaps, decide to just apply a basic cap functionality - when you reach that cap, it stops/disables all chargeable resources above the free-tier and gives you 72 hours to go raise the cap or they'll delete everything except things in their AWS Backup.
AWS can eat the storage cost for the stopped things without a noticable impact on revenue.
Perhaps a more advanced workflow-type setup:
When monthly spend reaches $x across all regions: - stop all running instances that are not below the free-tier limit - apply deny policies for per-request charged services - send an email alert to the billing, tech and admin contacts.
When monthly spend reaches $y across all regions: - Delete all chargable resources not tagged with 'Foo: bar' - send an email alert to the billing, tech and admin contacts.
One of the other things that would be great, is to be able to apply limits to AWS Accounts. For instance, the AWS Account that we give to developers to experiment on, it would be great to forbid them from starting baremetal instances, or GPU instances, etc.
Some of it can be controlled through deny rules, but not enough.
Billing alerts are also currently nowhere near responsive enough - it can be hours or days before you get told that some resource is running up your bill.
A comprehensive dashboard of 'here is every chargeable resource running in this account, in all regions, right now' wouldn't go astray, either.
Usually if you're a single person and you leave an expensive product running unknowingly, AWS support have been kind and refund you the money and take the loss as 'customer made an error' type transaction.
It's not like they hold the debt over your head for life.
I think the article is a bit extreme to highlight the person's reaction, it's pretty easy bait for most people to swallow. Realistically though, if you can't afford $20, how on earth did you afford a credit card to sign up for AWS in the first place?
There are people comparing this person's response with the Robinhood investor that lost money and took their life. The real issue here is a mental health problem and better awareness around that - AWS is a cheap scapegoat we can all blame to ignore the real issue at hand...
There's a big problem of: There's a huge amount of AWS that even if you configure it "right" - still leaves you with potentially unlimited liability.
Take a simple example of using Cloudfront and S3 together to host a basic website. There's a good well documented way of securing this all "right".
However that doesn't protect you from "slashdotting" - perhaps you put an image on your site and it gets linked from the frontpage of Reddit.
You go from paying a few $/month, to suddenly overnight having potentially hundreds or thousands of dollars of spend because tens of millions of people have viewed your image.
AWS Billing alerts are not going to save you, because once they (eventually, maybe a day or so late) fire, and assuming you immediately see that notification, figure out what it is being accessed, and remove that file (or just turn off the bucket/cloudfront entirely) - the bills could have gone way up.
It gets worse when you're talking about other AWS services.
The whole point being that billing caps, along with billing alerts that are actually timely, are actually necessary.
You shouldn't have to depend on the grace of AWS Customer Service to not force you to decide whether to pay rent, eat, or your massive credit card bill.
> Realistically though, if you can't afford $20, how on earth did you afford a credit card to sign up for AWS in the first place?
The example was $200. And for a lot of people, particularly students and other people new to development - $200 can be a large chunk of money.
Part of the problem is that you go to AWS Conferences and talks, and they talk about how there's all this amazing stuff, and encourage people to sign up and try it out.
What they never do is tell you "Oh, by the way, this demo costs $x/hour". I'd love to see some kind of taxi-meter style box behind the presenter/in the corner of webcasts showing how much the AWS Bill would be for the demo.
Taking a 4MB image, that would equate to around 8TB which is priced at under $1.
Make that 20 million views and you hit $10 - oof!
---
$20 or $200 doesn't matter - how on earth could you have a credit card if you can't afford to pay off $200?
---
AWS is great and people are free to try it out. They should also be aware that things cost money and not to put their credit card down for services without being fully aware of the costs associated.
It's like buying a property for investment then complaining to the real estate agent later down the line for a number of issues the property has that require maintenance down the line.
What's wrong with buyer beware? Why can't people/big companies offer services without people crying foul about 'getting hurt from bill shock'? Where's the personal responsibility and accountability for your own actions and decisions gone?
AWS aren't scheming to scape little bits of money from small time developers. They're interested in catching big fish companies who are looking to choose their platform / service offering in the cloud computing space. If you think these small time developers with 'bill caps so i don't overspend' is at the top of their feature priority list you've got the wrong idea about the business.
Who cares how they got the card, maybe it's a debit card, maybe it's their parent's or partners. Maybe they've got one that only allows a very low limit of charges.
The point is that there's a whole ton of people for whom $200 isn't a "Oh, well, that's a learning exercise - better be more careful next time".
> What's wrong with buyer beware?
Buyer beware is fine where all parties are fully aware of all repercussions.
However, you'll note that this doesn't apply to certain kinds of transactions, particularly financial transactions.
In some cases, you're either required to go through an expert who can advise you on the downsides and risks, or the seller is required to ensure that the buyer is fully informed of the risks.
It's the same reason we have warning signs on a whole bunch of things about how they could injure themselves.
Without the ability for someone to limit the damage, someone could be up for thousands of dollars in spend without being aware of how they even got there.
> AWS aren't scheming to scape little bits of money from small time developers
It's not even students and other inexperienced people that end up in bill-shock.
I've dealt with well-experienced senior developers who've gone off to an AWS Conference and spent a day or three being lulled by the story about how X is the new hotness, and would solve all the problems we have.
The AWS reps have all confirmed the story that X is the new hotness, and it's perfect for some new solution - and they've got some big name customer who's implemented it and saved millions Vs their legacy on-site solution. The AWS reps also hand out tens of thousands in account credits like they're mentos, with the message "Go play, see if it works for you".
Those senior developers have taken a look at the pricing on the thing, thought they understood it, and then fired up a demo to prove it out. When the billing starts to come in, there's been shock from them that they spent so much in so little time.
The pricing on things is quite opaque with one headline price for something, but doesn't include that you'll also need to consider the pricing for other bits. The billing is often delayed quite signfiicantly, and alerting is only useful as an indicator of how much you've fucked up, rather than a way to prevent it.
So, yeah, with AWS marketing all their solutions as being much lower cost, while at the same time not making it clear on what things do cost, and not making tools that you can prevent major fuckups -- they definitely do need to shoulder some blame.