If I was a foreign government I'd have multiple spies working inside Google by now.
Beyond that, the hard part of training a spy to get into Google would be getting a good enough computer science student involved. From there, it's really just a matter of teaching them to cover their tracks semi-intelligently. However, given what I'm sure is a mountain of completely legitimate reasons to look at user data (for example, to resolve data corruption, investigate malicious users, etc.) and an inconceivably larger mountain of user data to look at, I don't actually think it'd be that hard to get away with it.
Any support/SRE/developer access to a users' GMail mailbox would be logged and if they exceeded their authorized access by such as accessing a "public" persons email, They'd be fired pretty quickly.
I'm confident that Google is doing a better job than pretty much anyone else, but this problem is a more or less unsolvable one.
Edited to add that another interesting idea is that the people who man the DC's are actually pretty sparse (relatively few people for a lot of servers) so it's not inconceivable that one could trigger a failure on an important box, take down a replica of the figure's mailbox, swap out the drive for RMA and then do a quick copy. I bet this would be easy.
I guess my point is that no level of internal controls at any company can actually stop a determined government. If that were true, governments, which are much more paranoid than tech companies, would have eradicated spying a long time ago.
I do not disagree with your overall assessment, but this is not strictly true. Most good real-world security schemes don't follow the 'root is God'-model of Unix, and for good reason. It's perfectly possible to design a system where each operation performed by a "superuser" must be validated, or at least logged.
If Google were serious they should have brought out Bruces company Counterpane and put him in charge of security.
Its blindingly obvious to any one with even a basic knowledge of computer security best practice.
Security is hard, and it is even harder when any device on the internet is intended to be able to work with the system, and it is even harder when you operate one of the most valuable networks in the world.
And systems used by your TLA's to handle law enforcement access are not available to "any device on the internet"
As I said they should be set up to only talk over a private circuit to one other end point and also have proper hardware crypto gear that is external to the systems.
separating the extraction of data and applying the decoding probably should have been done on separate systems.
http://techcrunch.com/2010/09/14/google-engineer-spying-fire...
A spy would also be illegal and, more importantly, potentially very embarrassing politically. It should be clear to anyone by now, the US government doesn't care about cost or efficiency. And further, there are secret court proceedings for national security kinds of cases (of which there are literally hundreds at any given time), so there's no secrecy advantage.
The KGB back in the day had an entire department Line X that was dedicated to industrial espionage - Putin apparently was in this Department
I know a senior developer in British telecom who worked on the system that tracks every private circuit in the UK and she was being PV'd (positively vetted its called developed vetting these days) - as she had root access to this system same as being TS cleared in the USA.