You don't have to wait for Siri to respond and you haven't had to for a while. Just say "Hey Siri" followed by the command with no pauses and it will handle it just fine.
(side benefit -- as a result of this conversation I have discovered that its possible to turn off the 'activate siri via side button' mis-feature -- HURRAY! I can count on one hand the number of times where I invoked siri this way on purpose and on 100 hands the times its happened on accident. [Yes that does mean I almost never intentionally invoke siri])