General technology for enabling AI capabilities with LLMs and Generative models
github.com
github.com
If you use the image as input, you can try the DONUT model [1] that directly answers questions without external OCR. It can be fine-tuned to suit your needs.
You can get a pretty decent reading of screens and text documents with Amazon Textract. They find key-value pairs pretty reliably.
Or an agent that can work on the Bash command line and can e.g. fix apt-get installation errors or network problems.