Hacker News Reader

> "I got hit with a huge AWS bill last month that wiped out most of my remaining runway"

This is exactly why I don't use AWS. I need 100% predicable billing.

Lately I have considered looking into AWS again, since their system is just leagues ahead of competitors.

Mind explaining how this "bug" caused such a massive bill? Did you not have alerts set up? No protections against scaling? Etc.

View on HN

I'm using API Gateway, Lambda, and the IOT Platform to do realtime lookup / enrichment stuff for contacts in our CRM.

I setup a webhook in the CRM to call the API when a contact was created or updated, which would then enrich (update) the contact. I didn't realize that I had to disable the option to not trigger the update when the contact was updated via the API. So it created an infinite loop of updates that called everything millions of times before we realized what was happening.

I guess the one positive here is that our system can scale and handle the traffic :)

Talk to AWS support and see if you can negotiate it down, explain the situation and they might be able to cut you a break so that you don't have to shut everything down. You have nothing to lose here, make sure you've at least attempted this first.

Reply on news.ycombinator.com