Detecting duplicates is not a solved problem; it's hard. You have to compare strings and everything.