Yeah...wish there was a better way than just opening the app and clicking the voice button. I haven't found one. My talking is usually in the car, so it's not too inconvenient for me to do that.
Can only hope google relaxes their limitations on choice of ai assistant. In the meantime I'm setting up a raspi with openwakeword to just act as an interface for voice to voice interactions for while I'm home.