Hacker News Reader

Comment by general_failure | Hacker News Reader

Not sure the title is correct. After all, the people use WhatsApp through Linux (Android), Symbian or whatever and not FreeBSD.

View on HN

The servers were (are?) FreeBSD.

The servers are FreeBSD 9.x. After Facebook acquisition they've tried migrating it all to Linux but quickly discovered that Linux cant cope with that amount of traffic and they decided the FreeBSD is the tool to be used here. That lead to 'famous' Facebook job ad looking for Linux kernel engineers to help them bring the Linux TCP/IP capabilities in par with FreeBSD. Currently they're testing FreeBSD 10.1-RELEASE for their next platform OS upgrade.

Do you have any idea why Facebook would rather develop a TCP/IP Stack for Linux to match FreeBSD's, instead of further evolving FreeBSD's stack? Licensing issues?

While I have no actual knowledge why they made that decision, its not very hard to guess, and the possible answers are around their own staff familiarity with FreeBSD, the pros of having the platform simplified by using single OS (Linux in their case, since for sure they've more servers with Linux already than they've acquired with WhatsApp) and all possible security and financial implications following these factors. However, it seems like at least for know they might be staying with FreeBSD a bit longer, what's a great PR/recognition win for the FreeBSD itself. I really doubt its for 'licencing issues' because (religious wars aside) the BSD license is way more attractive to businesses than GPL one.

Said all that - I might be terribly wrong about everything I said, because its not an insider knowledge ;)

How could licensing issues be a problem?

I think people prefer a monoculture, even though the security idea of having multiple software stacks is appealing, thoroughly testing issues on two OS stacks is a lot more work, especially at Facebook scale.

Presumably they are tied into Linux for a lot of their internal infrastructure.

The main security threat these days isn't viruses or worms so a monoculture makes no real difference.

In fact running multiple operating systems is more likely to lead to a misconfiguration issue which allows an attacker into your network.

If someone getting into your network is your threat model then, well, you have already been owned. If you want to have resilient services where you can swap them out if there are security issues, a monoculture is not a win.

Facebook is almost certainly further evolving Linux's existing TCP/IP stack, not developing a new one.

I would rather guess it is for consistency reasons. People are more familiar with Linux than FreeBSD.

Similarly one of the reasons Facebook relies on PHP is because a lot of developers are familiar with it.

I know lots of people who are more familiar with FreeBSD who are working on Linux systems. The Linux monoculture is just massively dominant.

Pretty telling technical story and also flattering for the FreeBSD team.

This is not true. Facebook has never attempted to migrate them to Linux. If WhatsApp wants to convert to Linux, it is their decision. More than likely they will stay on FreeBSD as that is what they know how to run and a good way to kill a product is to force them to do things instead of focusing on building their product.

http://bsd.slashdot.org/story/14/08/06/1731218/facebook-seek...

A job posting in August indicates a failed migration attempt that couldn't have started until October, because that's when the transaction closed?

> Linux cant cope with that amount of traffic

I like FreeBSD but I don't think we can say that Linux can't cope with that amount of traffic while Google and Facebook use Linux. There is a missing piece here that's not about network traffic.

I think it is amount of traffic per server. WhatsApp didnt have a lot of servers and maybe reengineering for massive scale out on Linux doesnt make sense versus staying with what works.

Are there more details available about this? What specifically about the Linux TCP/IP stack is weaker than BSD's stack? Where does the weakness lie?

It doesn't have to be a weaker stack per se. It is also a lot about knowledge. WhatsApp vested much time into understanding FreeBSD. It may be if you vested the equal amount of time on Linux with knowledgable people, then you would see roughly the same performance.

If I should take a guess, the advantage of the FreeBSD stack is that it is simpler and easier to understand. Hence, it is easier to tune for performance.

Yes, of course, and Whatsapp users care very much about what's going on behind the scenes ;-) I'm not a mainframe user because I book flights, or a Windows user because i tried Bing.

It takes very little sense of humor to detect the fact that the title is a nerdy joke ;)

Reply on news.ycombinator.com