Yes, the law is muddied, and yes, there are many, many exceptions... most notably those as defined in Terry v Ohio (ie, Terry stops), but even with Terry stops, officers need to have a reasonable suspicion that you're either presently armed, or should be considered dangerous enough to warrant a stop.
It isn't extremism to interpret the law as it is written, and as it is considered by many courts to be valid, that requires police to need at least some mandate to being pulled over (whether a traffic violation, exigent circumstances) more specific than "this guy was black", or "this guy was poor", or what have you.
The problem is that with qualified immunity and deference, we allow the police to get away with a lot of stops that don't even meet the legislative standards we've set, and as we allow these things to continue, from one generation to the next, the standards for what is or isn't a valid stop gets lowered.
TLDR, if a cop pulls you over just because he can, and he cannot formulate a reasonable suspicion, then he has overstepped his boundaries in the eyes of the law, even where that law seemingly has bent over backwards to accommodate arbitrary stops.
And it's backed by the 4th Amendment, which is neither a law nor a rule. It's an acknowledgment of an innate natural right that all human beings have which includes the right to be left alone except when there's reasonable suspicion or probable cause.
I threw the thing in about skin color not because I have any reason to believe you are racist, but because we are talking about cops, who in many areas have been shown to be quite racist, and have been shown to pull persons of color over far more than white people.
[1] https://www.injurytriallawyer.com/blog/washington-state-dui-...
Or in other words not be harassed. This is also pretty well covered by the fourth amendment.