Hacker News Reader

Comment by pinewurst | Hacker News Reader

It's interesting that Dell won this one. Sadly there's not a lot of technical info about the system itself or the big storage expansion (probably more Lustre, sadly - my tax dollars at work).

View on HN

But would GPFS really be any better? What other competition is there for big iron hpc distributed storage?

Lustre nominally scales higher than GPFS, but GPFS is more reliable and better performing for anything having to do with metadata. It's also a lot more expensive license-wise. There's other competition, but it's not proven at these scale levels. Top 500 deals are bad business so it's not really worth it to develop/test for them thus leaving the business to the incumbents (e.g. the dumpster fire that's IBM or the Arkham Asylum that's Lustre).

FYI I work directly with the amazing guy who wrote this: https://csc.fi/web/blog/post/-/blogs/the-largest-unplanned-o...

Lustre at scale is a disaster, but one you can pay for people to help you with. GPFS is unbelievably expensive, but is a lot more reliable wrt metadata, but it is also quite fragile.

In my original question, I mean other than the obvious one of Lustre.

I'm sure the GFS model will be better with a faster computer. We can get more ensemble runs, at the very least.

I was referring to GPFS as in IBM's commercial General Parallel FileSystem, not the Global Forecast System.

Didn't Dell acquire EMC?

Yes, but EMC doesn't do that level of storage. For HPC, Dell resells various hodgepodges of their servers running Lustre (or rarely GPFS) with JBOD or cheesy RAID for the disks themselves.

To be fair, the post is not clear in whether Dell is supplying the storage as well. NOAA may be adding SAN storage from a place like DDN.

And DDN would still not be enough. DDN would be paired with something such as Lustre or IBM's GPFS to run the storage for big iron such as this.

I understand that, because those are basically the only filesystems anyone could use for this application. The comment was aimed at the "various hodgepodges of their servers ... with JBOD or cheesy RAID for the disks themselves" part.

Isilon is part of EMC which is part of Dell (now). They do scale-out fault tolerant storage up to multi-petabytes.

But Isilon is purely NFS (and/or SMB/CIFS) and doesn't support the parallel access that GPFS or Lustre do. It's not really a choice for this workload. Plus it'd be a lot more expensive than HPC storage - enterprise level RAS (reliability, serviceability, availability) costs. I've seen comparable capacity Isilon & DDN quotes (w/GPFS or Lustre) and they're worlds apart.

Reply on news.ycombinator.com