There's no sharp line. It's however people choose to define you. If they identify you as non-white, you get treated as non-white. Which means automatic negative judgments that wouldn't apply if they identified you as white.
That doesn't mean whiteness by itself gets you into every corridor of power. But non-whiteness excludes you from a lot of places, or gets you treated with suspicion when you're allowed to be there at all. It's a permanent set of bricks in your briefcase, on top of all the other ones that affect everybody.