Could you (assuming no care about efficiency)
Send the query to both GPU and CPU pipelines at the same time and use whichever comes back first
Send the query to both GPU and CPU pipelines at the same time and use whichever comes back first
GPUs can incur higher failure risks, and thus one will not normally find them in high-reliability roles. =3