Here is a concrete example where this mentality really hurt Google in the cloud space: App Engine. App Engine was really Google's first foray (or one of the first) into the cloud space. When it was first released, the only persistent store available was the Datastore. The Datastore was/is a great piece of technology, but it was painfully clear that this tool was optimized for Google's use cases and came from the mindset that things need to be infinitely scalable/shardable, e.g. eventual consistency and the fact that you couldn't have indexes with inequalities in different directions, so doing a query that is trivial in a DB (e.g "dateCol > someMinVal AND dateCol < someMaxVal") was annoyingly difficult.

So the early days of App Engine were really cool, but it took some time before Google actually made a plain ol' database (their Cloud SQL tech) available. As someone looking in, and as someone who has talked to a lot of ex-Google employees, it was clearly a cultural shift for them to need to focus on standardized enterprise tools that so much of the industry is familiar with, rather than all the tons of custom shit Google has due to their scale.

There are a series of jokes in Google about things like this. Things like just wanting to serve a file, or run some small webapp, etc, etc. Google's tooling is geared towards building large, reliable apps. That makes wanting to build some tiny thing seem like a big bother (seemingly lots of overhead).

I remember my first hackathon at Google, probably in 2015. My group built a service and deployed it with some automated deployment tools. In "production", it was running 9 instances, spread over 3 continents. A little overkill for a hackathon project.

But I have never heard it (in 7 years) seriously.

A running gag at Google is that the infrastructure makes hard problems easier and easy problems harder. There's a new-hire parable centered around the phrase "I just want to serve five terabytes" that demonstrates all the layers of systems and services you have to plug into to get a simple static-data server running reliably, scalably, and with monitoring and support.

The parable hints that "The Google way" to solve that problem is to find the service someone already built and use it to vend your data; don't re-invent a wheel. But (a) that means part of your job is now keeping abreast of all of Google's wheels (and there are hundreds of them) and (b) if your work didn't require the creation of a new wheel, was it impactful enough to justify a promotion later?

I'm an outsider, but this book[0] talks about how they think about these questions. The mentality makes sense to me in the sense that they are also trying to scale development processes. Does it make sense for every team to start small with a single VPS running python scripts and then migrate to up to 3-tier architecture, then all the way up to k8s when the feature is a success so needs global scale and availability? Or could the company make k8s easy enough that its trivial to just host v1 there, and back it with a massive multi-tenant db that also works fine for low volume prototypes?

0: https://abseil.io/resources/swe_at_google.2.pdf

No, like the phrase "I don't remember how to count that low", I've pretty much only heard it used ironically to poke fun at how infrastructure can complicate simple use cases.

Want to host a 1PB dataset, globally available with enough failover to handle a moderately large nuclear war? Easy! Want to throw up a 200KB web page with best-effort reliability? Surprisingly complicated.

It's a bit tongue-in-cheek, but there's some truth in it at least for anything infrastructure related. If your thing is a success, it almost certainly needs to scale to Very Large numbers, and be reliable/distributed/performant/etc. So if you're proposing something that you expect to be successful but only needs a couple of VMs, it would be an outlier.

(I work there)

Of course not. Don't be silly. The post even tells you with its title: "I don’t Know How To Count That Low.

:)

I've heard someone from Google talk about how they only think about the "B's and T's" of their projects. B's and T's, as in, Billions of users and Trillions of dollars.

Personally I've never heard anybody utter something similar.

No, and that informs my estimate of the accuracy the other judgments in the story.