1. Hallucinations
2. Prompt injections
Currently, there is no known way to detect either using LLMs themselves. As a research assistant, if the LLM hallucinates, and it always sounds extremely confident when it does so, the LLM itself is of little use and causes additional burden, defeating the whole point of this.
Maybe an external validation step that employs a pagerank like algorithm is needed to detect and flag hallucinations? If so, how valuable would that company be?