With Apple Watch that's always listening, what's at stake is public perception

Even if it’s technically legal, it’s the court of public opinion that will matter.

I could see certain businesses banning them, or offices not allowing employees to wear them.

I think I will think twice before having certain conversations with people if I see an Apple Watch on their wrist.

AFAIK, in the US it's fine (as in legal) if only one of the two people talking is aware that the conversation is being recorded. Which is different from what Siri AI does — it transcripts the audio, only. No idea how it this will be accepted as proof.