That happens with all products built on top of Open AI API. There are a few tricks to avoid this.

Each book is best at answering questions on its own source material. For now though, it will sometimes answer stuff not related to the book.

The litmus test I use is always asking a new book "Do you know anything about the Chamber of Secrets" -- often times will say no. Sometimes will return the most semantically similar thing it can find.

This will be fixed eventually!