Hacker News Reader

Comment by throwaway0asd | Hacker News Reader

I live really close to their largest data center and only about 500 people work there. It’s 4x larger now than when they initially broke ground and even then it was insanely huge.

Edit.

I just saw the photo for their Nebraska data center, so I guess each of their locations are about the same size and shape.

View on HN

>only about 500 people work there

I'm actually surprised there would be that many. This report [1] from the US chamber of commerce says 157 after initial construction and I've seen numbers lower than that. (Some may be a function of presumably outsourced workers like security guards and janitorial services.)

Datacenters bring money to an area while they're being built but there are very few local jobs once they're operational. (Obviously engineers do work connected to datacenters from other locations.)

[1] https://www.uschamber.com/sites/default/files/ctec_datacente...

> Datacenters bring money to an area while they're being built but there are very few local jobs once they're operational.

I really wish local politicians would get this. Here in Norway, several small communities got wooed by businesses wanting free land and cheap electricity for data centers, in return for promises of hundreds if not thousands of jobs from all those supposedly using the data center.

One even ended up having to buy back land they gave for free[1] for ~$700k.

[1]: https://www.nrk.no/nordland/ballangen-ga-bort-tomt-for-a-fa-...

You don't need people in a datacenter. You only need a skeleton crew for hands when things physically break, everything else should be done remotely

So what then is the argument about them needing more employees to serve more users?

The frontend apps are all the same, whether they have 3 users or 3 billion.

The front end apps are not all the same. If you built an app that handles 3 users well do you think it would easily handle 3 billion with no tweaks at any layer (app, framework, OS, network, hardware, edge traffic routing, internal dc network fabric, caching layer, storage for binary/ graph/ KV/ relational, cluster scheduler, security )? Another example, ML backed products (eg. Search, any RecSys) requires tuning features based on product specific needs.

You also end up leaving massive efficiencies on the floor by not hiring more specialists as you scale. This is a good read which explains it in detail https://danluu.com/in-house/

I would also suggest reading their published papers on in house systems to understand the challenges that come with operating at a large scale.

https://news.ycombinator.com/item?id=30940595

I’m sure FB could operate with a range of employees 40k-100k(?), but that number scaling with users is far from surprising.

You've mostly explained backend-related issues, no?

Yeah, most of that is backend issues. Part of them dictated by product requirements eg. We require serving 5 year old, <4MB photos with 99th percentile latency of 1 second.

Product and UI has its own complexities, but it seems clear each product is unique and would require different logic.

Then at most 3 people per product frontend would be enough? But the real number is more like 300+, so I can see the parent's point.

When your datacenters are huge, things physically break constantly.

What size crew do you infer when thinking of a “huge data center”

I’m still thinking it’s just enough to repair failures in a timely manner (and assuming there is spare capacity so you are not blocked if something breaks). Which means only a few hands on deck and the rest being triaged remotely.

Or not maybe we are still in the 90s when you need someone sitting at a desk in a server room to handle password resets :P

I'm in the same area, and agree the scale is dramatic. One land surveyor a friend ran into has been working there for 7 consecutive years. I think if I were in that role, I would want to see different environments from time to time.

Reply on news.ycombinator.com