There are many programs out there that will find duplicates files. I use this one. It gives you the option of replacing the duplicates with either hard or symbolic links.
From the project:
> This name is not as bad as it seems, because I was also thinking about using words like żółć, gżegżółka or żołądź, but I gave up on these ideas because they contained Polish characters, *which would cause difficulty in searching for the project.*
Really dodged a bullet there.
They work shockingly well on large libraries, I've managed to check millions of files consuming several TBs in a matter of hours.