The bots are not a solution, they're table stakes. Withholding that information would not do anyone any justice.
For the readers, it also strips the information of a lot of nuance. I obsess over word choice for a reason. Then there are the footnotes, the citations, and the purpose-built tools embedded in the content.
As I said, I also lose access to the community and its feedback and encouragement.
That's now, before tech companies start throwing ads on top of the information it hoovered from my website. I'm glad that for a while, destroying the independent web will create value for the shareholders.
The premise behind libraries being public works rather than, say, private collections is that society as a whole is benefitting from this information being free and creating that information as well.
Here you are limited by the whims of a private actor which can be one way one day, and another way when VC money runs out or when it has a "fiduciary responsibility to shareholders" which is the inherent conflict the original poster is sharing.
Having all of the worlds information being parsed by private actors or non-profits who may become private at some time is a problem and the anthesis of "information wants to be free"