Your recording is also likely to pick up the conversation of the people nearby saying "hey look at that".
EDIT: It would also probably be illegal to, in the course of taking your vacation video of whales breaching or whatever, instead start specifically recording a conversation happening nearby. IANAL, but I think the distinction has more to do with what you intend to record than what you happen to record. You don't have a "reasonable expectation of privacy" in public, but you do have a "reasonable expectation" of not being deliberately and specifically recorded by some random creeper while out in public.
EDIT 2: Further, with as often as it happens, and pursuant to the DoJ's own statements on the subject, the police don't have a "reasonable expectation" of not being recorded when they're going about their jobs — particularly when, in the course of executing said job, they end up beating someone into unconsciousness...
Are you outside your house? If yes, expect to have all your actions recorded and available to be streamed on the internet.