The google network is gold plated, lacks the jitter inherent in the internet at large or inside other competitors' networks. It makes it tempting to ignore some aspects of distributed computing, if only for a moment.
a: dark fiber IRUs between cities/metro areas
b: N x 10 and 100 Gbps wavelengths as L2 transport services from city to city, from a major carrier such as level3 or zayo
c: some combination of A and B
and they use that to build backbone links between their own network equipment that they have full control over. Google is its own AS and operates its own transport network around the US 48 states and around the world.
the exact design of what they're doing within their own AS at layers 1 and 2 is pretty opaque unless you happen to be a carrier partner that is willing to violate a whole raft of NDAs. But basically they've built their own backbone to a very massive scale yet without the huge capital expense of actually laying their own fiber between cities.
their network has incredibly low jitter because they don't run their links to saturation, and know EXACTLY what the latency is supposed to be from router interface to router interface between the pairs of core routers that are installed in each major city. Down to five decimal places, most likely. When you have your own dark fiber IRUs and operate your own WDM transport platforms you are in possession of things like OTDR traces for your dark fiber that tells you down to four decimal places the km length of your fiber path.
It also helps that the sort of people who have 'enable' on the AS15169 routers and core network gear are recruited from the top tier of network engineers and appropriately compensated. If they weren't working for Google they would be working for another major global player like NTT, DT, France Telecom/Orange, SingTel or Softbank.
The B4 paper states multiple times that Google runs links at almost 100% saturation, versus the standard 30-40%. That's accomplished through the use of SDN technology and, even before that, through strict application of QoS.
https://web.stanford.edu/class/cs244/papers/b4-sigcomm2013.p...
A few more details about strategies here:
https://research.google.com/pubs/archive/45385.pdf
Then there's a whole bunch of other host-side optimizations, including the use of new congestion control algorithms.
http://queue.acm.org/detail.cfm?id=3022184
You might recognize the name of the last author...
Though you do need to define "saturation". Are you referring to bulk bandwidth or some other measure of throughput/goodput? Saturating in terms of raw bandwidth can reduce useful throughput due to latency issues.
As contrasted with what would happen if you had a theoretical hosting operation behind 2 x 10 Gbps transit connections to two upstreams, and tried to run both circuits at 8 to 9 Gbps outbound 24x7.