Those sound like stop gaps at best. Its pretty clear the intended goal here. APIs are easy to integrate with but most systems in existence only have a visual interface intended for humans.
The end goal here is clear, being able to interface with anything available in the screen.