You remembered more details of the story than me but I manged to find it... 3 years after someone else asked about it on SE:
This is Randall Garrett's "Time Fuze", first published in IF, March 1954.
https://scifi.stackexchange.com/questions/203584/short-story...