Of course this mostly stands out in the federal government, which is almost certainly wasting money by maintaining so many separate law enforcement agencies, especially when a certain degree of standardization is in place in that most federal law enforcement officers (but not all) train at FLETC. This makes you question whey there are a million separate agencies instead of one large one.
And for the most part the answer isn't really a good one... just an agglomeration of history, with the most significant part being a result of the Inspectors General Act which basically created the situation that every executive branch agency has its own internal law enforcement organization.
For example, given that the Federal Protective Service exists, one wonders about the necessity of so many executive branch agencies having their own law enforcement unit to protect their facilities. But then one wonders why the Federal Protective Service exists when there is some historical justification for assigning that duty to the Secret Service. And so on.
Look at it this way: a similar controversy has existed at a local level over university and school district police departments acquiring ex-military equipment and establishing tactical units. Once again it's probably a lower risk than the municipal police department, but it does say something about the state of militarization. And then one can wonder why it is typical in many areas for universities and school districts to have their own police departments, which often face funding, training, and retention challenges compared to the municipal departments... Around here the university police departments have a reputation as a mill that pays for people to attend the academy, after which they spend their mandatory six months and then immediately transfer to a municipal agency with better pay. As you can imagine this does wonders for the quality of the workforce in the university departments.
More broadly, this whole thing also touches on the uneven training and staffing standards for US law enforcement agencies, which seems to result in good part from both the sheer number of such agencies and the various peculiarity of the state laws under which they obtain their authority (e.g. consider states that do and do not commission security police). This should be less of a problem with federal agencies due to the de facto standards established by FLETC, but we've seen in e.g. Portland that the inconsistent policies of federal agencies with regard to oversight, use of force, etc. can very much be wielded against the citizenry.