Imagine that you have a non-session-persisting load balancer in front of a cluster of web servers (n >= 3). Since this load balancer doesn't direct requests in any persistent fashion, a user making subsequent requests may be directed to any of the servers available in the cluster in a non-deterministic manner. In your application, you need to handle image uploads from users, and as soon as the upload is finished, you need to show the user their uploaded image. This creates a problem for you because after an upload, the load balancer may direct the user to a web server which does not have the users recently uploaded image resulting in a 404 Not Found for the image request and a poor user experience.
In the current system, an rsync process runs every 5 minutes and copies uploaded images around to each server so that every server has every file, eventually. This isn't a very optimal solution because users may not see their uploaded images for up to 5 minutes depending on which server they are connecting to for any given request and it forces every server to have a copy of every image uploaded, which may not be the most scalable solution.
Assuming you could not modify the load balancer, how would you redesign or fix this system so that files are uploaded in a way that it doesn't matter which server in a cluster a user connects to in order to be able to satisfy a request for the file? There are many ways to solve this problem. Please describe one or more ways and discuss the pros and cons of your solution(s). If your solution is very simple (not a bad thing), consider offering a couple alternate solutions so we have a strong understanding of how you evaluate and approach problems.
-
My Response:
Basically that this was poor architecture and didn't make sense. They shouldn't be using their web servers as a file store. They didn't like that.
In case we can't add any new servers, I would save the initial web server address (ex: lan ip) where the file was uploaded, to the database (we are using a common database, right?) and when the image is requested (and doesn't exist on the current web server), reverse proxy from the initial web server.
Those architectures are a little hack 'n' slash, but you got to do what you got to do.
It might well have been a hypothetical problem they've had in the past or that they've heard of, or it might have even been that their application requirements evolved into the use case above.
Not answering the question doesn't indicate that you can solve the problem. Right or wrong, businesses arrive at scenarios like this often due to circumstances outside their control.
Either way, I like it as an interview question, as it would effectively filter out people who are either not clever enough to solve it or not motivated enough.
When I hire, one of the best things I can hope to find in an potential employee is a 'can-do' attitude. While that might sound cliche, you can hire pretty much any smart person to point out the problems you have. It's much more difficult to hire somebody who can both identify those problems and come up with potential solutions and is capable of fixing them.
More research and planning from the start, saves time and money in the end.
In this case, centralizing storage makes sense.
Pros: less stuff to manage, no more rsyncs, immediate feedback on all servers. Save money by only needing to buy 1/n amount of space (assuming the space required is significant) because you're only maintaining X GB centrally versus X GB * n servers.
Cons: Money up front, increased latency to fetch data, but you're already waiting up to 5 minutes.
Quizzing people on language features seems a tiny bit silly, but I respect it as a method of ensuring expertise in a given language.
edit- I actually don't think it ensures expertise, but it can be a signal of a lack of expertise, so it is a useful method to avoid a false positive on a hire.
Here's my take:
My first inclination would be to think about all the gas stations in my neighborhood, and all of those I pass as I travel somewhere. The gas stations start to spread out the further from civilization I go, but I've never gone more than a quarter to half a tank of gas before seeing another. (I have a phobia about running out of gas in the middle of nowhere.)
But the difficult part it is thinking about how gas stations are set up. There is definitely a relationship between population density and gas station count. The less people, the less gas stations -- but conversely there seems to be a point where the population is so saturated (SF proper) that gas stations are scarce inside the city.
But maybe I just think that because of the relative distances I'm traveling. Inside the city I'm walking and see less gas stations, but outside the city I'm driving and covering a longer distance in the same amount of time and see more gas stations.
Also, we seldom see huge lines at the gas stations which means they are plentiful enough to service all available commuting cars without significant queue. If there are just enough to service cars without a queue, maybe we can say this (naively):
X number of people in the US. Three people per family, so X/3 = Y number of cars. (A lot of families have more than one car, I know) If each gas station has an average of Z pumps and is utilized constantly at an average of W% through the day, we need at least Y * Z * W% gas stations to service everyone appropriately.