Are there any LLMs or AI apps that allow me to give them a book such as "The Lean Startup" and I can ask them questions like "What are the arguments and what is the evidence?" without having to worry that it might hallucinate?
I'm not sure if there's a length limit, but basically you upload a PDF and it'll try to "ground" the LLM in that.
> A key difference between NotebookLM and traditional AI chatbots is that NotebookLM lets you “ground” the language model in your notes and sources. Source-grounding effectively creates a personalized AI that’s versed in the information relevant to you. Starting today, you can ground NotebookLM in specific Google Docs that you choose, and we’ll be adding additional formats soon.
(It now supports PDFs and plaintext, from GDrive or elsewhere)
------------------
In my personal experiments, though, it didn't really seem to be very different than just uploading a file into ChatGPT (in a custom transformer). Sometimes one or the other would be higher-quality; both would still possibly hallucinate. Importantly, neither one is strictly limited to the source text; they both still have prior outside knowledge.
What was cool, though, was NotebookLM's ability to cite passages in the source. Here is an example of your question:
https://share.cleanshot.com/pSnz4CXC
Their UI is a bit buggy, and sometimes questions just... disappear. That's why in the screenshot I asked it the same thing twice.
But still, might be a "good enough" way to get the gist of a new source that's not already part of the training set.
I heard this method works great for using new documentation, needing to combine multiple api's, and wanting ai to handle the coding.
1. You'd only have to know basic web dev. Enough to get input from the user, send it to an API (e.g. https://openai.com/blog/openai-api), and display results. There are many tutorials online showing how to do this.
2. You'd then design a prompt ("prompt engineering") that tells the GPT it's read the book, asking it to converse in a friendly manner as an expert literary critic, etc. Place this prompt and the prior conversation before the user's new question. Then send it to the API. (It's likely the GPT at least knows something about most popular books.)
3. You can now try to improve the results. A popular method: If you have the full text for the book, you can do a text search for the user's query. Then just include relevant parts of the book in the prompt.
I am imagine something more complicated, like having a chatbot with the personality of the character in the book answering hypothetical questions. Like asking gandolf why he didn't send the eagles to drop the ring!!!! Or asking dumbledledore why don't he create horcrux himself and fight voldemore, since he was able to defeat him once at the ministry of magic.
So here is kind of my understanding of pre gpt3 like models like BERT
1. Bert or any sentence transformer models generate embedding on the entire book (search space)
2. You pipe in your query to the same model generate embedding (query)
3. you do ANN or bruteforce KNN (lsh, pq) on top of the search space embedding with your query, essentially finding dot product with lowest value
What I am having trouble understanding is using sentence transformer does not give you answer using the character of the book, but LLM does.
How do I build a chat app that do that? Do I just use openai api? Or can I train my own llm or use off the shelf llm like llama?
https://chatfai.com/characters/book
I haven't tried it, but if I had to guess how it is built they're probably just setting up RAG vector databases at a per-book level and then augmenting a given character's context window with information from the vector database relevant to the conversation.
It would be relatively trivial (weekend project) to roll your own using streamlit + quant/pgvector + ggerganov llama.cpp and a suitable model such as Vicuna/Mistral/etc. Hardest part would be separating an entire book into a well representated set of embeddings.
1. Use sentence transformer, transform the entire harry potter or lord of the ring book into embeddings
2. transform query into embeddings -> "why don't gandolf sent the eagles"
3. Find most relevant text using ANN through the query embeddings
4. pipe in the context + query to llama
However the result is not very good, am I missing something in RAG?
Experiment with a "sliding scale" around the book (paragraphs, pages, etc). Try to use a graph to relate book sections, etc.
Consider setting up a tuner with well defined questions and answers to search for optimality around embeddings.
I think my problem with making something myself is finding a vector db that works on the AMD chip and Windows platform.
free for now