It would be (relatively) trivial for congress to decide to make it so that "rounding up every red haired and freckled person and putting them in a concentration camp" would be legal.
The question isn't legality, per se - it's whether we believe that it follows the values and principles that we want our country to have.