> and being illegal isn't the same thing as being a crime
I think in a very strict interpretation of what "lawful" and "criminal" means, this is likely true. There are civil infractions and there are "criminal offences." But most people, colloquially, equivocate "illegal" with "crime." And if the police engage in unlawful (illegal) activity, they can face penalty. So I'm not entirely sure what you're trying to say. Someone would, of course, need to file a complaint in a court of law, but that doesn't mean that they were "permitted" to do it and can't face penalty as a result of ignoring their lawfully mandated restrictions.
Also recognize, and disclaimer IANAL, that it does not require the Supreme Court to rule on a constitutional violation. The Supreme Court, as I'm sure most of us know, do not have the capacity to take all cases that are presented to them. First a chain of appellate courts must rule after which the Supreme Court becomes the last available avenue, and they typically only accept the "interesting" constitutional challenges... where "was this a constitutional violation or not?" has been deemed ambiguous.
But "lower courts" can, and do often, rule on whether something was a constitutional violation.
So I guess the point is, the police cannot engage in constitutional violations with impunity, but of course in order for them to face penalty a complaint would have to filed and it would need to be taken to trial and the police found guilty.