What's scarier, that you saw an ad regarding very personal details of your life because your phone was listening, or even though your phone wasn't listening?
People see this happen and reach for an explanation they understand, "my phone must have been eavesdropping!" The truth is these ad and data companies know this about you without your phone having to listen.
To twist a quote from the Matrix: "So you're saying my phone listens to me? No, I'm saying these companies have such wide insights into your personal life and information that they don't have to listen."