Can you share some data on costs and scalability?
At Kadoa, we're working on fully automating unstructured data ETL from websites, PDFs, etc. We quickly realized that doing this for a few data sources with low complexity is one thing, doing it for thousands of sources daily in a reliable, scalable, and cost-efficient way is a whole different beast.
Using LLMs for every data extraction would be way too expensive and very slow. Instead, we use LLMs to generate the scraper and data transformation code and subsequently adapt it to website changes, which is highly efficient.