Hacker News Reader

Comment by smilesnd | Hacker News Reader

Netflix uses bsd because it has great IO handling.

https://www.quora.com/Why-did-Netflix-choose-FreeBSD-over-Li...

But all the supercomputers use their own custom linux. So no commerical backing. Also these computers are not your standard data center. They cut networking and storage to a minimal because those are bottlenecks. These things are just massive ram/cpu/gpu boxes connected properly through pci.

Edit: I was looking at Sunway hardware specs the number one supercomputer they use a PCI-E 3.0 connection for all there nodes. Communication between the nodes is 12GB/second with a latency of 1 us. Their total ram is 1.31 PB

View on HN

But all the supercomputers use their own custom linux. So no commerical backing.

This is just wrong. Yes, they use custom Linux, but it is highly highly supported. You buy a Cray or a BlueGene and you get dedicated kernel engineers as well as on site support etc etc.

They cut networking and storage to a minimal because those are bottlenecks. These things are just massive ram/cpu/gpu boxes connected properly through pci.

This is just wrong. Networking is extremely important in supercomputers - but it isn't like setting up a LAN. They use custom networking, Infiniband, Aries, OmniPath etc. There isn't much information about the "PCIe Network" on the Sunway, but the fact it is PCIe isn't very interesting - everyone has fast optical networking. It's the topology and protocol which makes things interesting.

I don't consider it commercial linux because they are not competing with other options. The companies that do build these supercomputers have to provide technical support because nothing out there exist for it. Just a different view of what commercial linux is vs building hardware specific software.

Its very much commercial linux, because you are paying for a service, that's linux based.

Sure with how cheap inifiband is (especially compared to 40/100 gig ethernet) one _could_ cobble together a system your self.

Where the magic sauce comes in, and where the like of cray really make things shine is the software they provide to allow end users _easily_ do multi-machine scaling.

libraries for just in time delivery of data directly into ram? yup. location aware job dispatchers that co-locate jobs near each other logically? yup.

All of those hard things are solved for you.

Redhat is a commercial linux because they are competing with other os/distro in this market. If I pay Joe $5 a month to keep my ubuntu up to date it doesn't make ubuntu a commercial linux even though I am paying for a linux service. These companies building supercomputers are competing in producing supercomputers. Not in providing a linux disto and providing a service for said linux. I very much doubt I could get access to their linux disto and linux service without first purchasing a supercomputer from them.

This is pretty much exactly how every HPC OS has been sold since the Cray X-MP. It's like if you buy Isilon - it is a software, hardware and support you buy. No one argues that isn't commercial.

The Quora discussion doesn't seem to add much. Just a guy saying 'FreeBSD is rock-solid' and '[for] raw performance, .. nothing beats FreeBSD', without giving any technical details.

Reply on news.ycombinator.com