That's only google, Siri doesn't work that way.
From Apple employee: "only collects the Siri voice clips in order to improve Siri itself"
Which is basically what Google says.
EDIT: both companies have promised "off-line" processing, whatever that means, but I don't know if any already does that. Note that off-line processing does NOT mean they won't store your voice clips on their servers.
I am afraid that soon it is hard for general public to avoid the situation where when they flush the toiled some unexpected device will recognise it and report to the headquarters.