Yes, but QPS vs. "queries to the API". The difference is the time slice. I should have been more explicit. The key here really is the time function between the numbers. That the AWS blog calls out trillions of API calls isn't relevant because there wasn't a specific time denominator. The 126M QPS is the important stat.
Parts, yes. In reference to the specifics mentioned in here though, those services run on Infra Spanner, not Cloud Spanner, but they're the same stack. The main reason things like Gmail, Ads, etc haven't swapped into GCP is because of the internal tooling that's built up around the infra spanner relating to those services specific to Google that don't make sense in Cloud Spanner.
Tell me more (bonus points if you find me on LinkedIn or other social because tracking comment responses on HN is really rough). I'd love feedback you have so I can bring it back to the product team!
Really glad to hear! Please find me on social media or LinkedIn and let me know how it goes for you using the PG layer. I'd love to hear more feedback.
You can! Spanner has a free trial: https://cloud.google.com/spanner/docs/free-trial-instance. Keep in mind, that per-request pricing isn't free unless you stay under the free tier. So just take a look at what those limits are because going above them means you're not free anymore.
The DeWitt clause is only there to prevent badly written performance data from getting attention it shouldn't. If anyone writes a good benchmark (good process, not good results, necessarily) to bring to us, our product teams absolutely will consider it.
More specifically, infra and cloud Spanner are the same stack. So they've progressed together hugely since 2013. :) The real differences between the two are more about the internal tooling we (Google) have around infra that's built up with our other services that consume it over the years that aren't relevant to anyone other than Google.
But that's kind of a moot point. I mean, if you're even looking at the likes of DynamoDB or Spanner, it's because you need the scale of those engines. PostgreSQL is fantastic, and even working for Google, I 100% agree with you. Just use PG...until you can't. Once you're in the realm of Spanner and DynamoDB, that's where this discussion becomes more of a thing.
100% this, and even though I work for Google I absolutely agree. BUT, for the folks that need it, PostgreSQL just DOESN'T cut it, so it's why we have databases like DynamoDB, Spanner, etc. Arguing that we should "Just use PG" is kinda a moot point.
True, but one would hope that both sides in this case would be putting their best foot forward. Getting peak performance out of right sizing your DB is part of that discussion. I can't imagine AWS would put down "126 million QPS" if they COULD have provided a larger instance that could deliver "200 million QPS", right? We have to assume at some point that both sides are putting their best foot forward given the service.
Infra and Cloud Spanner are the same stack. Having those services run on infra is more about the legacy of tooling to shift it rather than anything around performance or ability to handle it
Infra and Cloud Spanner are the same stack. Having those services run on infra is more about the legacy of tooling to shift it rather than anything around performance or ability to handle it.
Just as a hand in the air...Be careful about what you're comparing here. # of API calls over a period of time is...largely irrelevant in the face of QPS. I can happily write a DDOS script that massively bombards a service, but if that halts my QPS then it doesn't matter. So sure, trillions of API calls were made (still impressive in the scope of the overall network of services, I'm not downplaying that), but ultimately, for DynamoDB and Spanner, it's the QPS that mattered to us in terms of comparisons of DB scaling and performance.
My PM also reminded me, depending on the data set size too, if it's bigger than the buffer cache, we might see some improvements over PG as well from that (we've made memory improvements around that which might help potentially).
I'm assuming you mean "I start using this under a free dev license, and want to shift up to the paid production version"? And what that looks like? If that's the case, I don't know yet. That's literally being discussed/hammered as we speak, and we likely won't have a good answer until we're ready to go public preview (not sure on timeline right now, depends heavily on how the tech preview goes). This stage of the game is literally a "We built an awesome thing, please poke at it and tell us where it does/doesn't work!".
The improvements we've made on the read side of things likely won't affect the geo data directly, BUT, depending on what other aggregate data you're combining with the geo data you might see some improvement from the columnar engine. Hard to know without digging in deeper on the queries themselves and schemas you're working with. I don't want to steer you down a path of moving a ton of data and infra just to look. Best guess is that it'll be a bit better, but likely closer to your PG experience vs. the BQ experience.
It really is, yes. I can't go into a lot of detail on the "why" because it's not open source and the product team would murder me...but I highly encourage folks to try it for themselves. Nothing else convinced me until I did it for myself. :)
As a heads up, we're unlikely to make the codebase for it open. I might be able to convince the product team to open source some components of it (our GM has talked about this before in a couple articles), the whole thing won't be open sourced.
No worries on the negativity! I totally understand. No one likes to have time wasted. :( I'm getting the more detailed info on the cache use-case to the eng team just so they're aware. Also, the can't disable it...so funny story. You can delete the cluster...save the $$$ on the AlloyDB side, and recreate it later and the data's still safely there. So the disable instance is 100% valid feedback (and has been raised before, I'll add your voice to it as well). And there's a workaround....sorta.
We don't have specific details yet about licensing or pricing. Stay tuned as we get closer to GA. Having said that, we're pushing hard for a free usage tier and the product team isn't pushing back. :)
We encourage folks to draw their own conclusions, but we have done comparisons and ours is more efficient and produces faster results. I only say test yourself because I can't give specific details of how/why, so don't take my word for it. :)
Not for Omni no. End-users are responsible for any HA/DR/Replication needs. Cloud AlloyDB has all of it built in, but since it was so tightly tied to the Cloud infrastructure it was simplified for Omni.
Code improvements. Indices aren't taken into account at all since that's so workload dependent. Having said that, the index advisor can make finding the right indices to improve performance there more easily as well. Vacuum management handles figuring out the right time to do garbage collection for you, etc. None of that is part of that 2x improvement.
Yeah, I know. Apologies. Stay tuned though! For the tech preview launch we really focused on a golden path (narrow). We'll be expanding compatibility as we march towards GA.
It's twice as fast as out of the box PG for most things, and up to 100x faster for reads, depending on what you're doing. So there's that.
Also, from a manageability, on top of the index advisor, there's also vacuum management, so it will figure out when the best time to do the garbage cleanup while minimizing impact on performance.