Hacker News Reader

Comment by thrownaway2424 | Hacker News Reader

Please throw out an estimate of how big you think a mirror of all Google's network traffic might be.

The NSA's Utah data center was designed to host more storage than existed on the entire planet at the time. Sheer volume of data is among the least of their concerns.

OK, so you can read wikipedia, but does it make any sense? A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon.

Remember when IBM announced they'd built the largest filesystem in history? That was only 120PB. A YB is 10 million times bigger than that.

Consider also that a YB is 1000x the entire storage industry output from 2012. So under your theory there is a parallel hard disk drive industry consisting of at least 99% of all hard disk production on Earth.

Facebook's servers spend most of their time finding, sorting and then transmitting bits to the billion people who use Facebook. Storing a bunch of data and storing a bunch of data that needs to be dynamically assembled in real time for a billion users are two very different things.

Regardless, no one has theorized that the NSA storing yottabytes of data. The (still probably crazy) rumors are zettabytes.

"no one has theorized" that except for Wikipedia, Wired magazine, C-Net, and most of the people commenting on this thread.

"NSA to store yottabytes in Utah data centre" -- http://crave.cnet.co.uk/gadgets/nsa-to-store-yottabytes-in-u...;

>" A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon."

You're off by a factor of 10-15 on the square footage. Per the wiki page, "The planned structure is 1 million or 1.5 million square feet."

"According to USACE, the center will have 100,000 square feet of raised-floor data center space and more than 900,000 square feet of technical support and administrative space."

Typical government installation. 90% overhead.

http://defensesystems.com/Articles/2011/01/07/NSA-spy-cyber-...

The NSA wouldn't be storing all the photos or videos from Facebook. Storing text is trivial compared to that.

If you reduced Facebook's storage demands down to always static text, I think there's little doubt you could eliminate over 90% of their storage requirements (I'd say closer to 97%). And that's without any kind of compression.

According to the Wired story on it, it's four 25,000 square foot data center buildings[1]. That's smaller than one of Apple's smaller data centers. Apple's Oregon one is going to be 500,000 square feet.

It's not going to be able to hold more than an average size datacenter, and there are a lot of data centers out there, each holding more data that would have to be copied. The NSA has huge capabilities, but they're not nearly there yet. Most of what we should be worried about is their legal capabilities.

edit: it's also worth noting that the wikipedia article's source is the wired article, and all the wired article says is that a DoD report says that they need to start designing their network to handle yottabytes, which isn't even related to the NSA's data center. Wired then does some interpolation by using "yottabyte" to make the requirements of the data center sound big, but they never say that it was designed to handle that level of data or that it will ever be able to handle that level of data (in fact, they never even cite any level of data that it was designed to handle). But the Wikipedia article cites the Wired article, and suddenly it's become "a data storage facility for the United States Intelligence Community that is designed to store data on the scale of yottabytes".

[1] http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/al...

[2] http://appleinsider.com/articles/12/08/16/apples_oregon_data...

> It's not going to be able to hold more than an average size datacenter, and there are a lot of data centers out there. T

Why not? Most data centers aren't designed to store a bunch of stuff, but instead are designed to serve a lot of users. Facebook has at least one storage forward data center and it packs an exabyte into a 60k sqft footprint:

http://www.datacenterknowledge.com/archives/2013/01/18/faceb...;

I don't think the NSA has zettabytes at hand, but you can store a massive amount of stuff in 100k square feet.

I was replying to the fantastical claim that the datacenter "was designed to host more storage than existed on the entire planet at the time". Of course 100,000 square feet is a medium sized data center and can hold quite a bit of data. If you're happy to go even slower, you could fit even more in with a tape archive that big. You're still so very very far away from a data center sucking up all human-produced data (even accounting for duplication) and storing it for later retrieval and analysis as many here apparently believe.

So yes, the NSA does have a new datacenter. But no, it's not the new Area 51 nor is it host to magic computers.

Big. But the NSA has been sucking in ridiculous amounts of data for many many years.

http://en.wikipedia.org/wiki/Room_641A http://en.wikipedia.org/wiki/Carnivore_(software)

the social "who's talking to who, and what sites did they visit" information... oh, probably this big -> http://en.wikipedia.org/wiki/Utah_Data_Center

I'd like to point out that their upper capacity limit, "a trillion terabytes", would require approximately 1 trillion hard drives, probably at a cost of $10-$100 trillion. I feel comfortable saying that they have not built out to that scale yet.

They don't need to access to all network traffic, just the interesting bits, like e.g. the internal traffic for the storage/backup systems used for GMail or Google Accounts .

And this is relevant to what?

It's relevant because some of the things that are being suggested here on HN are clearly infeasible, from an infrastructure standpoint.

Word. I get that. I just thought the question sort of steered away from the main point of the parent comment, which was that if Google did have some sort of nefarious partnership that they would be legally obliged to deny it no matter what. My apologies, I'm probably just being a bit nitpicky.

O RLY?

http://en.wikipedia.org/wiki/Utah_Data_Center

I think his point is it would be extremely expensive. The leaked presentation claimed the cost of PRISM was only $20M.

PRISM is clearly not the whole picture. The Utah Data Center alone is much more than $20M a year and PRISM's specified purpose was using US IT infrastructure to intercept foreign communications, which would presumably be filtered from a large data set.

The bulk of the data transmission and storage costs are probably under another budget

Reply on news.ycombinator.com