My layman's understanding is that any barrier will stop some percent of most things. In the case of small, airborne particles, this stopped percentage of already-dry, floating material is very small. That means inhaling through them is relatively ineffective at stopping you from inhaling the particles.

When you exhale it is more wet and the moisture causes the barrier to catch more of the airborne particles. The barrier also catches any more-solid material that would otherwise land elsewhere, providing a mechanism for the particle to spread.

This means the net gain for wearing them to keep from catching the disease is relatively small compared to the gain by wearing them to prevent the spread if you may have been exposed.