At some level, you have to accept that things like Siri operate just like talking to human customer service representatives, who these days always begin conversations with a rote "This call may be recorded for quality and customer service purposes."
What it comes down to is whether you trust the company to be unreasonably diligent in controlling what happens to these recordings. If there is a criticism to be levelled here, I think it is around the question of how well Apple and everyone else in this space allow consumers to make informed choices.
Fine print in a voluminous user agreement is not nearly the same standard of disclosure as "Your words may be recorded for quality and customer service purposes" every time you say "Hey Siri." The latter may be impractical, of course, but perhaps there is a middle ground. It's not a dichotomy.