Comment by Mawr | Hacker News Reader

Comment by Mawr | Hacker News Reader

Here we see the most classic interviewing error: not understanding that there's a difference between a test and what's being tested.

Will the data fit in memory? Well, of course it will, it's an interview... oh you expected me to ask you anyway?

I should obviously load both files into the hashmap, that way it works for an arbitrary amount of files instead of just two... oh, you expected me to write a solution for literally the exact problem you stated without considering its practicality? Even though before you wanted the opposite, when you asked about the algorithmic complexity?

Guess I'm failing.

An interview could reasonably ask for an out-of-core solution.

But you are right that optimizing specifically for two files seems wrong, especially as the data contains a timestamp, so you could simplify the problem and ignore the number of files completely.

You can generalize to

> Now, given two N log files we want to generate a list of ‘loyal customers’ that meet the criteria of: (a) they came on ALL days, and (b) they visited at least L unique pages.

while keeping linear time complexity and

  O(num records in first file * L)

memory complexity. It is not too different from the solution given in the article (just use 3 maps instead of two).

That means that multiple files doesn't need out-of-core if the maps for one file at a time fit memory.

Reply on news.ycombinator.com