Yes, it would be a challenge to make it work, and work fast! Lots of numbers to crunch! But it's definitely possible, you could e.g. use idle threads to collate data in the background while not handling requests, and serve results leveraging the collated database.

The simplest way would probably be to cluster the users according to their preferences and serve your results according to which cluster a user belongs to. More complex mechanisms, like detailed preference correlation graphs might be possible, but would consume huge amounts of storage. But I really fancy this idea, and I do seriously believe it to be one of the next big breakthroughs which are going to happen.

Unfortunately I do currently not have the time to handle such a big assignment, so I decided to throw the idea out there, free for the grabs. I want to have that thing, and I don't care much who's gonna make the big bucks from it!

But try to convince anyone that an openly touted idea is actually worth something. People still seem believe that if you're not secretive about your ideas, they probably suck... :-(

Another one: Right now some people are talking about so-called "niche social networks", i.e. concentrate on users within a certain niche, which are automatically much more homogeneous in their interests and preferences. HN is actually kind of an example of one. I'd rather have one social network for everyone which automatically sorts the users into niches (clusters), where the same user might fit into multiple niches.

The big advantage is that the networking site will find the people with common interests for you instead of the users having to find (and tend to) the adequate niche networking sites!