Anderson made an argument against the SSC https://www.the-scientist.com/opinion-old/the-case-against-t..., which I pretty much agree with. Science funding is finite and physics talent in a country as well. Many really good students are funnelled into dead end careers in high-energy physics (whether theoretical or experimental). It's just a huge waste of human potential, especially given the fact of how ruthlessly they are exploited. I know people in the field, a hiring decision between three people was recently described to me as a choice between a 'social case' and two competent workers, one of which happens to be a friend of mine.
Funnily enough lot's of institutions doing fundamental research in high energy physics either also do military research or receive military funding. Most of Witten's work for example has been funded by the Department of Energy. The whole reason CERN was build in a neutral country was because people worried that a post nuclear arms race would break out otherwise. In France one of the major institutes contributing to particle physics (Saclay Nuclear Research Centre) also developed their nuclear arsenal and is located next to a major arms manufacturers research center (Thalys).