Does anyone have experience applying these models to rendered content (PDF's, webpages, etc). Seems like a really promising area of research to achieve LLM agents.
It's not perfect, but it works: https://github.com/OpenAdaptAI/OpenAdapt/pull/610
> the most recent latent space pod
It works surprisingly well despite the fact that the model was not trained on this type of data.