Great observations! Thanks for your deep dive into result. I didn't go into this level of detail myself, but one thing I notice is that "the cat" in the graph is actually Peter, the cat that Tom gave painkiller to (with missing connections to Tom and Aunt Polly).

You're absolutely right that some characters are missing even in those short books, and there are likely many more relationships that haven't been fully captured. That said, I’m still quite impressed by how much data the LLM extracted in a single pass, especially given the complexity of the task, the size of the input, and the strict output format.

My estimate of quality was subjective. To truly quantify accuracy, we’d need to establish a "ground truth" with a better approach and measure the difference between the generated and actual relationship graphs. One possible way to do that would be to process the text in multiple passes: first extracting characters, then identifying relationships, both steps with more sophisticated prompt engineering. Another way is to manually annotate the network. The only book I found with a publicly available, human-annotated character network is Les Misérables, based on Donald Knuth’s work: https://github.com/MADStudioNU/lesmiserables-character-netwo...

However, there is an additional challenge. Even with human annotation, the question remains: how to define relationship network? What is a relationship in a book? Should it be limited to explicitly stated connections in the text, or it also can include deduced relationships based on context with some probability? Defining these criteria is crucial to quantify quality of the result.