I've seen this posted every so often. How are you intended to use it? Just click a folder and pick a random paper?
Aho-Corasick "Efficient String Matching: An Aid to Bibliographic Search", or Bentley-McIlroy "Data Compression Using Long Common Strings" are 2 such good places to start.