BigQuery can cost a fortune. Our data engineer messed up once by not time partioning the DB and it racked up a ~10K USD bill in a month due to the high number of queries that were running on it.
- I agree that, by default, the on-demand pricing model is pure consumption, and folks can and do mess up sometimes.
- I had a habit of refunding folks when they asked, and if they made obvious mistakes, even though it took me an average of 3-4 hours to process each refund.
- BigQuery has cost controls to prevent runaway costs, sounds like you must check it out ASAP
- BigQuery also has a DDL option to require users to include a partition filter predicate in queries
- BigQuery also has flat-rate pricing, on which the vast majority of folks above SMB on. There are no runaway costs with this one.
I think generally BigQuery is a great example of an extreme serverless consumption-model. You have thousands of cores at your fingertips, and, well, if you do something that overuses, you are allowed, but you do pay for it.
However, I agree its a really cool tool, I have not worked on the ETL side of things, but we use BigQuery in almost all of the projects I have been on, my teams usually needs to figure out the right query to determine key KPIs our clients usually ask for and the SQL is really helpful.
Interesting. Why?