What's the value of these scant few thousand unpublished romance and fantasy novels in the context of the rest of the corpus -- vast scrapings, all of Wikipedia, etc.? A sample of how people write? Why aren't more public domain works included?
The Pile (the 800GB dataset by Eluther AI) contains BookCorpus2, along with two much larger datasets of books (and a whole lot of not-book stuff). From their paper [0] the reasoning for the book datasets is that they are "are invaluable for long-range context modeling research and coherent storytelling". The reasoning for including BookCorpus2 next to Books3 and Project Gutenberg boils down to "no significant overlap with the other datasets, and others use it".
In general books are a great source of extremely high quality long-form content. They are longer than most content found on the web, and are generally of high quality, having gone through many revision rounds between editor and author. Just that both of these aren't really true of BookCorpus. Even a dump of highly rated stories from fanfiction.org might be better.