I believe it was already known that anything trained on The Pile contained references to copyrighted material from scihub. It seems unlikely that folks who chose to use these sources were completely unaware of the nature of the data. Presumably, given the urgency in the last 2-3 years to be a leader in this space, a number of shortcuts were taken.