You know what let me try my iPhone, maybe lowering the volume enough would do the trick indeed.

The issue is the speaker btw not the mic. I want(ed) it to be outside of my ear as in the bluetooth wearable headsets though.