Really cool project! I'm doing something similar at a very small scale for my personal project using TypeChat with Zod (https://github.com/microsoft/Typechat) and Unstructured (https://unstructured.io/)