This is basically a solved problem for supersonic ammunition (not fired from a silenced weapon): https://en.wikipedia.org/wiki/Boomerang_%28countermeasure%29 (not to suggest in any way that the proposal is practical on a wide scale).