Hacker News Reader

Hacker News

Comment by jamesfinlayson | Hacker News Reader

Parent Full thread

jamesfinlayson

Not sure - are Google running their servers with custom-made processors as well?

View on HN

A lot of it is just mundane details. Some of it leaks out in ABSL, for example some boring trivia is at:

https://github.com/abseil/abseil-cpp/blob/master/absl/base/i...

They also have for years had major patches to TCP which Linux refuses to adopt. Some details at:

https://www.ietf.org/proceedings/97/slides/slides-97-tcpm-tc...

These won't make binaries "not run" as such but they are necessary for the correct and efficient operation of large-scale distributed systems.

QUIC, Snap/Pony, and user-space thread scheduling are strictly superior to what happens inside the kernel and their development makes the delta between prodkernel and upstream kernel less relevant over time. They also, collectively, make Linux itself less relevant.

Linux kernel is the most sucessful free beer UNIX clone, but it is by no means the only one, and if the trends on IoT and cloud infrastructure are any measure, in a couple of decades we might still live under POSIX monoculture to some extent, but Linux most likely won't be the kernel underneath.

Not CPUs but a lot of other components are completely custom, developed in-house. A well known example: TPUs. Google is one of the world's largest manufacturer of servers and other DC equipment, actually, it's just that none of that is for sale (though a lot is for rent :) ).

Unless they've been building any factories I don't know about, their contractors are the manufacturers. I'm talking about companies based in Taiwan like Quanta, Compal, Clevo, and others. If you've ever been to a Computex Taipei you have seen the booths of the actual manufacturers of Google's hardware.

https://en.wikipedia.org/wiki/Computex

This is a meaningless nitpicking. It is like nitpicking Apple is not a manufacturer of hardware since the actual assembly happens under some contractor. For all intent and purpose, and even technically, Google is the largest manufacturer of data center-sized supercomputers.

See this for an early example: https://news.ycombinator.com/item?id=542531

Like when that Pluto Switch got leaked back in 2012

https://www.wired.com/2012/09/pluto-switch/

That's covered earlier (full quote):

> Those patches implement various internal APIs (e.g. for Google Fibers), provide hardware support, add performance optimizations, and contain other "tweaks that are needed to run binaries that we use at Google".

I'd assume if you're making allowances for custom-made processors at the kernel level you wouldn't also want that leaking into your user land binaries as well.

Why not? They could have kernel accommodation for one feature, and user-land leakage for another.

Also, for some features you need both kernel and userland cooperation. Just think of eg fuse or io_uring or mmap that are in the public kernel. You can surely imagine that Google might brew up similar features.

(I used to work at Google. But I didn't have any special insider knowledge about their kernel stuff. Which is good, so I can't violate any lingering NDAs here with my speculation..)

Kind of, I've seen some non-public Intel SKUs in the fleet but afaik they just had tweaked core counts and clocks for better perf/watt. They were also exploring non-intel architectures at some point, ppc especially but no significant prod deployment yet. And then I'm sure you've heard of special purpose hardware like TPUs

Some of them, yes! At least for machine learning. They also have custom silicon in their phones (and the phones run a Linux).

Reply on news.ycombinator.com