GPT3 was trained on 45TB of text. what would you estimate that is as a percentage of all extant discrete written text? or as a percentage of text written in the Roman script perhaps
If 2-3 mb of text on average, the Library of Congress's 24 million books — excluding the large amount of of manuscripts and other items would be about 60 TB of text for a reference point. Depending how you count what is available on the internet I imagine it may dwarf that number but I would expect a lot more redundancy and less information-per-word
there's also a substantial amount of text in the archives of businesses all over the world - i.e. operational procedures, mission statements, reports, etc. I can't even roughly estimate what that would be as a % of all extant written media though