Have you tried moondream yet[1]? The moondream 3 preview model[2], according to the blogpost[3] appears to outperform many frontier models on VLM tasks and does so with a relatively small footprint.

[1] https://moondream.ai/

[2] https://huggingface.co/moondream/moondream3-preview

[3] https://moondream.ai/blog/moondream-3-preview