In today's world, most phones or "smart" devices are also constantly listening; I want to believe they don't listen until the trigger phrase is uttered, which could also be implemented for these ultrasonic applications, but I'm not entirely convinced and them always listening is but a silent over-the-air update or setting change away.
They can't know whether the phrase was uttered unless they constantly listen.
The microphone is active and "listening" all the time.
The firmware that detects the wake word compares the constant input stream against waveforms that are designated "wake words". Firmware can be sometimes updated for custom or trained words, but it doesn't hold a large dictionary.
If a reasonable match is found, it kicks the full recording/recognition/streaming code, squirts any buffered audio at it (to catch words that come directly after the wake word and before the full handler is ready), and then things proceed according to plan. Depending on the device and service, recognition might happen locally or in the cloud.