I've seen a few people say 'google3'.
Q: is it actually the code that you loved, or simply the tooling that exists?
(and if it's tooling, why can't that type of tooling be replicated for other codebases outside of google?)
Q: is it actually the code that you loved, or simply the tooling that exists?
(and if it's tooling, why can't that type of tooling be replicated for other codebases outside of google?)
Bazel isn't the whole of everything though, the other piece being exported is kubernetes, which isn't Google's borg, but that's its roots. There's Apache Airflow if you need a workflow engine like Sisyphus, no shortage of databases to choose from, though now we're drifting into the operations side of things.
But basically, Google has invested untold millions of dollars in the form of SWE and other engineering hours into making Google3 operate. If anyone else invested that kind of dough, with enough smarts, they'd also be able to make it a good experience. The problem is few people have that kind of budget, and even fewer invest in that thing, preferring to use free tools instead.
What tool do you use to edit code, and how much did your employer spend on that for you?
After burning an entire weekend like five years back trying to get a released version of tensorflow to build from source [0], I'm catastrophically disinclined to use Bazel for anything.
I found its diagnostics to be utterly unhelpful, its documentation to say nearly nothing I needed to know, and the various Internet resources for the build system to be somewhere between "as confused and lost as I was" and "total fanboy who is so expert in the system that they are incapable of speaking like anything other than an architecture astronaut".
I'm sure it's legitimately fuckin amazing when you learn it at a company that's big enough to have one or more entire teams dedicated to internal developer tooling (and training for and documentation of the same), but (at least in my experience) for those of us on the outside, it's just bad, bad, bad.
[0] The fucking thing wouldn't even build in the officially-supplied "build tensorflow" Docker image. I was utterly unable to find out why. I get that this indicates that the tensorflow folks fucked up somehow, but the fact that I was utterly unable to figure out how to understand WHAT they fucked up is pretty damning.
We’re having incredible build performance with it, and I find its language much more readable than CMake or Make. We’re also using Bazel for things like packaging.
I would encourage you to give it another go.
I spent two entire days attempting to understand (let alone correct) whatever was going wrong with the Tensorflow build.
The error messages were inscrutable, as was the debug output. There was a fair bit of official documentation, but little of it was "unhappy path" documentation, and what there was was unhelpful. As far as I could tell (and it was difficult to tell given how I was generally unable to understand the problem), none of the folks talking about this tool on the Greater Internet had ever run into this problem. [0]
I've never run into this problem with CMake, SCons, Make, or any of a dozen or two other build systems I've worked with. To put it another way, I've NEVER had a single problem with a build system be utterly impossible to understand and solve inside of a working day. (Sure, I've had to spend multiple working days solving multiple problems to get a build right, but I've never had any one problem prove to be utterly unsolvable in one day, let alone two.)
How easy it is to get into failure modes and (more crucially) how much support the system and supporting documentation gives you for both identifying the failure mode you're in and getting out of it are much, much more important than being the fastest thing out there. [1][2]
Unless I'm getting paid to use it and am required to use it and nothing else, Bazel is absolutely not worth my time.
[0] This must obviously be not the case. Given the circumstances, whatever I was running into, it's certain that TONS of people must have run into it. But it's telling that I couldn't discover any other people who were running into this.
[1] Yes, I'm sure that you've never had a significant problem with the tool. There are many, many useful tools out there that folks rave about that have burned me and burned the folks I work with badly enough that we won't touch them, ever.
[2] Also, if you learned to use Bazel inside the Googleplex, then that absolutely is a scenario covered by my statement: "I'm sure it's legitimately fuckin amazing when you learn it at a company that's big enough to have one or more entire teams dedicated to internal developer tooling..."
E.g., Codespaces, CoPilot, Actions, etc?
(genuinely curious, not trolling)
Codespaces is like CitC; I haven't been a Google since the rise of AI so I can't comment on how the internal equivalent to Copilot is; Actions is very primitive compared to what Google has, but yeah, you can see where it's going. it doesn't compare right now but it could, eventually.
I still mainly do my LLM assisted coding in chat-style interfaces that are more like pair programming by mail.
I think the Gemini code assist is coming out as a consumer product, I forgot the name, because Google is awful at branding.
Why not? Is it related to partial checkouts?
I think the reason its not easy replicable is:
1. It takes a ton of initial investment and ongoing maintenance but its worth it when your code base is gigantic.
2. There is a consistent set of top down enforced rules. With the consistency it becomes much, much easier to build tight integrations between tools.
(almost?) everything is buildable by a single build system (blaze). When anyone can consistently build/test/run anything in your codebase it becomes a lot easier to build a whole host of potential tools like code search.
Probably someone can dive deeper than I can. But one thing I learned the most important property for a code base to be maintainable/scalable is consistency.
Things like on team rosters, on call rosters, code review permissions, service permissions, feature flags.
All of it stored in google3 and can all utilize the same consistent set of tooling like presubmit test, integration tests, deployment automation, permissions, code search.
Its sort of like Infrastructure as Code but more.
As someone who hasn't worked for Google, how does Google's implementation of this differ from e.g. Guix/nixpkgs? Being able to easily build/test any available package is a big reason I like using tools like those.
I think the public bazel (public version of blaze) docs is probably the best place to look.
The elegance of the tooling from what I hear is that there's tons of different tools maintained by different teams that work seamlessly (and fast) together to produce google3 and all of its supporting pieces.
But to answer your question, sure it can. But good luck building your own. Google has been doing this since the 2000s.
And if you're a big company already, you've already bought into your existing patterns & design choices; things like that are VERY hard to change.