Hacker News Reader

Comment by cplusplusfellow | Hacker News Reader

Where is this extra code coming from? Do they do an immense amount of code generation?

Googler, opinions are my own.

I believe the YouTube video linked from the wired article goes into this, but there's more machine generated code checked in than human generated code at this point. A lot of this is boilerplate or config data that is stored in source control.

I feel like Google definitely believes that config is code, so we have a ton of configuration of our systems checked into Piper.

Why is the generated code checked in, vs generated as-needed later?

Most of them are generated on the fly, like protos.

But you know, there are tons of configs, all kinds of.

Traceability, simplicity, all sorts of reasons.

If you commit all your codegen’d stuff you don’t need to worry about how to run codegen in most places. You can enforce a lot of security properties, you can run large scale analysis on the code in production because it’s all “statically” available, etc.

That said, I think Google’s source control ecosystem is really quite different to what’s publicly available with systems like Git/Hub/Lab etc, even to Phabricator. These approaches work well at Google but don’t necessarily translate to other tools in the same way.

Don’t take these sorts of things as gospel, or even as google advice about good engineering. Understand what they achieve, and then figure out a way to achieve that same result with the tools available to you.

Pure speculation on my part, but I've noticed Go has generate and build as separate steps and it's standard to check in generated code. This split no doubt makes compilation a lot simpler/faster, but perhaps it's a reflection of standard practices the language authors were used to.

Theory: it makes secure builds that much more straightforward. You can just point goma at a repo and whatever it inhales is everything needed. The simplicity cascades downwards.

For example, "everything comes from a repo" would make object (.o) caching that much more straightforward.

And you can reason both about the logic behind a service and how it was configured from a single commit hash/CL.

Sorry, I never checked the replies to this comment, but I just want to point out that I was just joking with my 10B statement and have no clue as to how much code is actually in google's repos. it may be 1T for all I know.

(Opinions are my own)

> Do they do an immense amount of code generation?

Blaze (aka Bazel [0]) has provisions that make it easy to generate code but this happens as a compile step rather than something that is checked into a git repo.

[0] - https://bazel.build/

300K person-years....

That would still be 20-30k additional LOC per person year. Which is difficult to achieve especially in a corporate setting with a mature code base, without code generation, lots of blank lines, or including static data. Even in startup environments it's hard to get 20k-30k a year for several years from the average unless many of the engineers don't overlap.

I don't know about that. 100 lines a day, 280-or-so days a year, is pretty doable for the average L4. You could do that in migrations alone, and people do.

L5 or above there's normally a lot more communication and planning, which pulls down the mean significantly. In migrations you typically need to subtract some old code in the end, and it's not uncommon to result in net negative LOC (if the migration is ever complete).

For L4 maybe there are more days where one can do several hundred LOC in an afternoon than L5, but there are also many days that result in 100 or less over an entire week, because there's testing, debugging, profiling and figuring out what to code.

But, maybe I'd be wrong about Google. Afterall, it might be easier to dump lots of code into the codebase if there are 10 independent chat apps in development at any one time.

Migrations tends to involve deleting lines too.

A 200 line change typically doesn't grow the code base by 200 lines.

I believe in my 2.5 years at Google, I've actually deleted more lines of code than I've added.

> 280-or-so days a year

So working some week-ends and no vacations at all?

There are about 260 week days in a year. Add bank holidays and vacations and you should ve closer to 230 working days in a year.

Probably half of that is config or autogen, perhaps even more, so it seems fairly reasonable?

Yes but the point of contention was whether this can be done without code generation and static data. And probably both you and I agree it's quite difficult for a large organization to get 20k-30k net LOC additions per engineer without "conifg or autogen".

Reply on news.ycombinator.com