The idea is that at some point they (being the first line) succumb to infection and can be come the site of a persistent infection that in turns causes more sore throats and re-infections.

At least that was my experience. I was advised to take them out in my mid 20's. It does seem that there is correlation, now that they are out I get less colds and sore throats.