One of the major problems is removing duplicate or near duplicate content like images, text etc....