I'll answer your question and then engage in some shameless self-promotion. I wrote about it after posting my comment. Here's the chunk which answers your question:

Where am I now?

Heh. I mentioned that looking for duplicate files is harder than I thought it would be. I'm actually on my third try. The first one was when I thought "I can do this with a script", the second was with .NET, where I aimed bigger, but found not nearly big enough.

So, I've just completed the work on the file crawler, and the next bit is submitting the crawl results to the index. I've done this part before, and I don't expect it to be particularly hard, but I have to find the time for it. After that, something resembling a UI (I am trying to solve a problem), then put the whole thing out there with a big fat "alpha" disclaimer (probably Apache license, since I'm using so much of their stuff).

And that's what I'm doing, and where I'm at.

Hope this helps.

http://itgoon.blogspot.com/2009/08/wheres-that-file.html