Yeah speed is definitely a weakness right now. One option is to switch the models around and try a faster one, that will probably give you some speed-up at the potential expense of accuracy. You can do that in the .skipperrc file. There's some gains to be made for prompting most likely, as well as in the Omniparser model.

A more future thing I'd like to do is have a model specifically for the action selection task which could be a much smaller network. That would really improve the latency.