Hacker News Reader

Comment by rdoherty | Hacker News Reader

Sigh, another completely synthetic benchmark with big iron: A dual Intel Xeon X5670 with 24GB of RAM from SoftLayer. The X5670 has 6 cores @ 2.93 GHz, 2 threads per core, /proc/cpuinfo shows 24 CPUs.

Serving a plain HTML page over localhost via nginx. A medium length blog post with pretty much no real-world information.

View on HN

Still, for comparison a huge site like reddit is serving around 1000 pageviews per second. The hardware in this post isn't that good, nowadays you can do quite a bit better. If you could get even 0.02% of these pageviews per second out of more modern hardware that's already 100 pageviews per second. Only a handful of sites need more than that. For example HN certainly does not. And AFAIK HN runs single core on a language (Arc) that compiles to Scheme, not the fastest in town. Since reddit is only about 70x larger than HN, we are probably close to the point where with a lot of optimization a site like reddit could be run on a single large box (if we're not there already).

Dude... I am sorry, but your reply does not make sense on many levels:

I don't know if reddit could run on a single server, but this benchmark does nothing to prove it. A pageview on reddit consist of several http requests. A http requests translates into several requests into a database backend. In this syntactic benchmark he is requesting the same static file over and over again, so nginx and/or linux can cache it in memory (or CPU cache).

Also even if they could put reddit on a single server, it would require a lot of optimization and engineering time is more expensive than hardware. It would also make adding new futures in the future a pain in ass, because you would have to be very greedy with CPU time (and other resources like memory, bandwidth, etc...). Also you want a multi-server setup for redundancy and handling extra capacity.

EDIT: I don't if you also planning to put the database on the same server? Web servers are basically stateless so they are easy to scale. It would be interesting to calculate how much disks that server would need just to handle the IO throughput.

Full disclosure: This is some speaking who used to work for social network with 9M users and 3,000 servers.

This benchmark does not, but the fact that Hacker News which is just 70x smaller but conceptually similar runs on a single core on a slow language does. I completely agree that it's probably not be a good idea, but the fact that it's possible or almost possible is still interesting in my opinion. If nothing else the power of modern hardware puts the obsession with horizontal scalability into perspective, since most sites are nowhere near the size of reddit.

Even though a pageview on reddit translates to several HTTP requests, only one of those requests has to serve dynamic HTML. The requests that get the JS, CSS and images can go to a content delivery network (and they probably already do). You'd probably indeed want to put the database on the same server, with the vast majority of DB accesses hitting main memory (as you can easily get a server with hundreds of gigabytes or even terabytes of RAM nowadays -- it helps that reddit's access pattern is very heavily concentrated on new posts). Of course you have to replicate to multiple servers to prevent data loss.

9M users and 3,000 servers sounds like a lot. What kind of servers are these? What is the size of the database, roughly?

That's a commodity 2CPU machine these days, and about a month from being dethroned by the new E5 Xeons.

Serving over multiple 10GE to front end cache servers physically colocated with it would be realistic -- not sure how much of a hit that would be vs. over localhost.

Static vs. dynamic content is a reasonable optimization to make for an application under heavy load.

That's a good point about the hardware. I was initially comparing it to cloud hosting where you'd most likely not have access to multiple cores.

Big iron? I guess if you've been spoofed into the cloud hysteria and you pay for an anemic node at the same price a dedicated runs.

nginx was probably using around 20mb ram, too. 24gb is ridiculously unnecessary, but it was probably the least the author could get with the rest of the specs.

Big iron? That server couldn't cost more than $5,000.

So it is a pretty obvious link bait for whatever company they are advertising.

Reply on news.ycombinator.com