Dealing with Data Deluge at LHC | Hacker News Reader

Dealing with Data Deluge at LHC

newscenter.lbl.gov

30 pointselorant15 comments

"generate up to 10 gigabytes of data"

That's got to be misprint .

Not all data is detector data. This is usually what we'd call Level 0 or Level 1 data. The lower levels aren't usually that interesting for analysis, because they might not be properly calibrated or even reconstructed, depending on the detector.

Reconstruction takes an immense amount of CPU time so it's not practical to do on the fly, so you have to store reconstructed data, and that takes up space. Then you might store (re)calibrated data and have several generations of that. Then you reprocess all of your data as a new pass, then you have several passes of data, constructed with different calibrations and algorithms.

Not mentioned is simulation (monte carlo) datasets. Their size is usually within an order of magnitude of your actual data size. For some experiments, the amount of simulation data may eclipse the amount of actual data they have.

I can't comment on the size and composition of ATLAS/CMS data, but my guess is the rate at which data is generated for the experiments, using a broader definition, is easily within an order of magnitude to that of Youtube.

https://en.wikipedia.org/wiki/Worldwide_LHC_Computing_Grid

thanks. "The data stream from the detectors provides approximately 300 GByte/s of data, which after filtering for "interesting events", results in a data stream of about 300 MByte/s. The CERN computer center, considered "Tier 0" of the LHC Computing Grid, has a dedicated 10 Gbit/s connection to the counting room."

Looking at the data rates of just the ATLAS detector [1] the data rate of about 10 GB/s for all experiments sounds about right (with higher event rates in the current run and detector upgrades). And that's after throwing away 99.99% of all events! The raw data rate is >64 TB/s (with about 1.6 MB per event)!

[1] http://www.atlas.ch/pdf/atlas_factsheet_4.pdf

I'm not sure if you think it's a misprint because it's too small, or two big.

As i interpret it, the article says there are up to 1B collisions per second, and at only ten bytes per collision you get 10gb/s right away.

in one second.

How long do the collisions last? Is this sustained over seconds?

IIRC, the beams intersect a few times in the 26km diameter tunnel, where the bunches go around at the speed of light. Depending on how many bunches there are, that's ~ thousands of beam intersections per second. How many particle collisions there are per intersection depends on how densely packed the particles are and how many of them there are, known as the "luminosity" of the beam.

And yes, in a "run", this is sustained indefinitely.

I imagine that the warehouse-scale computing people at Google (for example) wince as they read this article.

And more when they see this:

https://wlcg-rebus.cern.ch/apps/capacities/federations/

(Note that tier-0 is the only real CERN datacenter. Apparently it is split into two pieces linked by 100gbps fiber. Maybe that's explosion protection? Anyway, the external partners are some kinda SETI@home arrangement.)

Google, can you just give CERN an old datacenter and 2 FTEs worth of volunteers?

I've worked both at Google and at CERN Openlab, and if your impression is CERN's grid effort < an old datacenter and 2 FTEs, you're greatly underestimating it.

I've only got experience with the former, not the latter, so yes I was estimating. I accept your appraisal of my estimate.

I guess I just wasn't impressed with the tier-0 core count--no, it's that I expected LHC to be the earth's darling project and get whatever resources it may need. This article describes a struggle that sounds resource-related... And, gah, academic software < industry software. :(

> I expected LHC to be the earth's darling project and get whatever resources it may need

Yeah, the DC at CERN is constrained by the power it can draw from Meyrin (or at least it was, 9 years ago). And the rest of the grid is donated by universities (with some trade mechanism to donate the CPU/disk time back to them in the future). So I expect any of the big Cloud providers can beat it. It'd be lovely if one of them donated resources, in exchange for bragging rights.

Meyrin? Oh, hey, I know that lake. I've been to Lausanne... It didn't even occur to me to try to visit the LHC on the way... d'oh!

Reply on news.ycombinator.com