The documentation right now [0] is pretty light on how this works. I watched the IO presentation[1] on App Actions and it was also unclear.
While semantic intents exist locally, it is unclear where Google maps from a user action back to that semantic intent, and many of their examples involve online google services (like Google Search and Google Assistant). They also say the intent must be registered in a Google database.
Look at the slide within the video at 23:30 specifically.