With modern NLP models being extremely good, can't you bundle together all sites whose content embeddings are very close? Presumably there's an acceptable threshold somewhere that achieves a good tradeoff of false positives vs negatives?