I mean, are they wrong? By all conventional metrics, Los Alamos County is one of the healthiest counties in the country. Obviously there are major confounders, but if it doesn’t readily show in the data maybe zero concern is the right level.
I asked one nuclear chemist what he thought about the new housing developments at TA-21 (or whatever it is), the site of the milling facility. He said "Sure, I'd live there. Well, I'd take my geiger counter".
Los Alamos is higher elevation than surrounding areas, on top of sandy soil that drains to the Rio Grande. I'd be more concerned about the health impacts on the people in the watershed. Los Alamos County has one of the highest per capita incomes and education levels in the country (I believe it has the highest percentage of PhDs), and pretty much everyone there works for the Labs, either as scientists or support staff. I'd me more concerned about the health impact on neighboring communities such as San Ildefonso Pueblo, which is quite the opposite - one of the lowest income and education levels in the country. Santa Fe is also a short distance down the river. The reservations there have indeed raised concerns about the impact of pollution on them, such as the aforementioned battle over chromium contamination.
You can solve a lot of health problems by just screening people frequently... including offsetting pretty serious pollution.