When you're on speaker, most platforms have a noise cancelling algorithm to prevent Larsen loops between mic and speaker. Using headphones usually disables it.

Look for USB conference headset