They almost certainly aren’t listening to you https://mobile.twitter.com/RobertGReeve/status/1397032784703.... But the likely explanation isn’t any better imo.
People see this happen and reach for an explanation they understand, "my phone must have been eavesdropping!" The truth is these ad and data companies know this about you without your phone having to listen.
To twist a quote from the Matrix: "So you're saying my phone listens to me? No, I'm saying these companies have such wide insights into your personal life and information that they don't have to listen."
I'm willing to believe it's a coincidence combined with my brain happening to bring it to my attention; but I'd believe they're hacking my microphone before I'd believe in clairvoyance.
I don't subscribe to them listening either to be clear
I was at a social outing some years back at pub that I had been to a number of times before with some longtime friends of mine. Somehow we got on the topic of favorite candies from childhood. I mentioned Warheads.
I hadn't had one of those in two decades, and certainly hadn't seen any ads for them in a long time.
But still, the very next day, I was getting ads for them on social media.
Everyone I have ever asked about it has a similar story.
You could whip up some kind of explanation that a friend of mine searched Warheads later. Or maybe a nearby bar patron overheard the conversation and searched for the candies.
That just seems so unlikely, though. It was a miniscule detail in a larger conversation. But in terms of retail products discussed, it was one of only a few.
1. The device has to record the things used by the always on mics listening for wake words somewhere, without any other developers figuring this out and sharing how it is done.
2. Apps, using a background task, have to monitor those recordings, run speech recognition, and pick out key words, without anyone identifying this process.
3. The apps then have send those keywords back to the ad network, without anyone ever sniffing the network traffic and seeing this process.
There would need to be a background service that has access to the mic without using the official APIs (which show a mic notification) that somehow no one has been able to find for the 5+ years that people have been claiming this is happening.
The truth for all of these is probably that we are boring and predictable combined with some version of the Baader-Meinhof Phenomenon.
Most people blindly install applications with whichever permissions are requested, and the typical social media user won't ever be intentionally force stopping the app when they're done checking it, so they are always effectively running in the background -- presuming that some battery optimization hasn't kicked in. Facebook and others are very overzealous about their default permissions, and that inarguably, absolutely, 1,000% gives them the technical ability to do whatever they want with your microphone while the app is running, and their user agreement allows them to do anything they want with anything they capture from any source.
So, with all of that in mind, it is completely reasonable to expect that audio recordings could be hashed locally and sent as routine API data without it being at all obvious. I'm not necessarily saying that past research into these claims had the wrong conclusions, but you're definitely being very naïve and apologetic toward companies whose entire business models are based around invading privacy at all costs, and aren't afraid to collude[0] with each other to make those invasions happen without public pushback.
> but you're definitely being very naïve and apologetic toward companies whose entire business models are based around invading privacy at all costs
I'm not being naïve and I don't have a facebook for the reasons you mention. My problem is that you're making alot of assertions with no backing and suggesting something that is extremely unlikely is true, so the burden of proof is on you to show that it happens or, at least, how it could happen.
Right now, I think it is highly unreasonable that the social apps are using exploits to gain access to the microphone without the OS being aware. Not to mention the challenge of hashing messy, real world audio for keywords.
And as far as real world audio being a challenge, I think you might be slightly behind the times on the power of perceptual hashing. It's disturbingly accurate with very little context or effort.
I am broadly aware of perceptual hashing and its capabilities. Where I feel like you are underestimating the amount of work involved is in performing perceptual hashing without anyone noticing and then matching without anyone noticing. If you take the OPs example of “warheads,” where does the matching happen?
Are you suggesting the social apps are storing the hashes of everything people said and then sending them to servers to match against a list of perceptual hashes? Because then you’re just hashing the audio and sending it, which would be obvious. The other option would require a list of perceptual hashed to be stored on device and a local match performed, this would, again, be obvious because now the social apps would have to store a giant set of hashes locally.
For example, Roku and TVs use the former - they sample your content and send hashes off to a server for matching, this is identifiable in their network data and would, similarly, be identifiable in mobile app network data. Importantly, they’re sampling controlled data and can simply send an image hash every X-seconds, you can’t do that with the “warheads” example, you need to capture and send it all to catch the key words.
As to the obviousness of sending hashes to a server, I can think of a dozen ways to make that look like routine API data that's getting synced during a routine polling process. The app is a black box until data hits a networking layer, and it can be made to look like anything they want before it gets there. Would they need to be acting deviously to accomplish this? Of course. Is it in these companies' nature and history to act deviously? You bet it is!
As for the complexity of matching without anyone noticing, I think you may be overestimating the amount of data needed to accomplish this task. They wouldn't need to record full conversations or phrases, or even have any context at all. As you and others have noted, they already have so much contextual information about us, that all it would take is successfully capturing a few words once in a while to really fine-tune they're targeting.
Some very simple audio processing could easily chop up "probable" words based on any number of factors. It could throw out any recording that didn't meet some minimal requirements or seemed noisy. And at that point, you're just sending some strings to the server occasionally, no longer than the typical cookie.
And again, they already brazenly do this with Messenger and WhatsApp and everything else in the world that isn't Signal. So, that it's both technically possible and they have already invested heavily into such systems, makes this all a real possibility. At the very least, it's something that should continue to be investigated and actively monitored for, because even if the biggest of the companies aren't doing it, that doesn't mean nobody is doing it -- or doing it on their behalf.
Yeah, I've had one of those happen to me where it was just uncanny if it wasn't listening. I walked into work and my coworker was sitting there working and asked how I was. As I put down my stuff and was taking my lunch out of my backpack I was like, "I'm good, but my backpack zipper just broke!"
When I logged into facebook ~30 minutes later I had ads for backpack zippers. It was downright unsettling and I actually deleted facebook app from my phone over it. I still remember that unnerving feeling in the pit of my stomach when I saw the ad, as I never got any ads like that previously. I definitely was one of those people who thought the "your phone is listening" stuff was paranoia, but that made me at least reconsider if not agree.
Are you sure you don't have cause and effect backwards, and the "somehow" was actually one of you seeing an advert for Warheads earlier because they'd started advertising more in general or more at your group's demographic, then they were near in mind to talk about, then the ads were more noticable?
I don't think this happens because I don't think it's technically feasible. The hotword detection stuff that is always on ("hey siri", "ok google") is incredibly limited in vocabulary.
Maybe one day that will change, maybe it's even possible now, but people have been telling stories like yours since well before I know for sure it was technically infeasible to do it.
However, if I am half way through a movie and I want to find some detail on it, it inevitably shows up within the first few search results whether I'm watching it on DVD, a blu-ray or off a streaming service or broadcast TV. The streaming service I can understand, Blu-ray player could be talking to something I guess, the other two are harder to explain since I use an old DVD player that has no networking facility.
The other thing I didn't count was that google probably know I search for movie information regularly and bias the search results accordingly, which is more plausible than it knowing what movie I am watching by listening to it. But maybe they have something like that "what am i listening to" feature that used to be in google play music.
I guess we unintionally might be triggering a pile of information side channels without the voice recognition spying being foremost.
I was watching it as MP4s I'd pirated /years/ ago, on VLC, on a computer running Arch Linux.
I have lots of examples that I can hand-wave away as interest profiles, but I've thought about this one since and just have absolutely no idea how it would happen. If someone could explain how this happened without using the mic, I'd be really interested. Better Call Saul hadn't touched my browsing history for years, and elder law never had at all.
I think that really is the most likely explanation. Facebook and Google knows who your friends are, and if a friend is interested in Warheads, maybe you are too. So you start seeing ads.
One recent example was some random show a friend was telling me to watch. Later that night, YouTube started recommending videos about the show. I've tried to explain it logically, in much the same way a character in scooby-doo would say "there must be a logical explanation for all of this" when faced with a ghost.
Did I search for it, or otherwise leave any intentional digital footprint of interest? Absolutely not.
Its reasonable to think Google has correlated the two of us as knowing one another. At the very least: I have this friend's phone number and email address in my Google Contacts. Our phones were in the same room together, and I certainly have some Google Apps installed. Maybe they know via GPS, or maybe its same WiFi/IP Address. It is reasonable that my friend has expressed a digital footprint of interest in the show, and Google is now funneling the results over to me, thinking "hey you have friends who are into this obscure show, maybe you will be to"
Of course, Google does not communicate its reasons. Its a black box.
What I cannot explain is why it chose that night to make the recommendation. We spend a few hours every week together. My friend has been watching this show for a while, mentioning it maybe once previously on a communication channel I feel reasonably satisfied as being "secure". Then, they mention it in person, and Google chooses that day to make the recommendation to me. Its one thing to assume I may be interested because this friend is interested; what raises concern with me was the perfect timing.
The funny part is, maybe Google isn't listening. It literally doesn't matter. Everyone I talk to, in my bubble outside of tech, believes they (meaning, "Big Tech") are. So, either: Google should be listening, because by-and-large people seem to think they are, and still use their products. Or: Google needs to realize their black box magic is conveying the impression that they are listening, and if their goal is to build trust, saying they aren't listening isn't enough; they need to dumb down their algorithms, or at least explain it better.
Personally: I think they're listening. I suspect its unintentional. I suspect their systems have grown so big, with so little responsible oversight, that some data stream is getting fed into some classifier then into another algorithm, and in the end its influencing the profile it builds on google accounts for things like ads and recommendations. I suspect it isn't the smartphones doing it; I suspect its Google Home speakers, in combination with a less controversial temporal + geographical correlation between two people.
Sure, this all sounds crazy. If you reading this works at Google; prove me wrong. I want to be proven wrong, with emphasis on the word proven. Why does YouTube make the recommendations it does? Why doesn't it explain the signals it uses in the generation of a recommendation? It must either be because you're afraid it would expose the depth and breadth of signals at your disposal, or, the algorithm isn't sophisticated enough to relay that information. And, maybe, just maybe, you'll begin to realize how bad both of those options are.