One option is bidding with points: player A looks at the position and bids X<0.5 points for the privilege of picking the color. Player B either accepts or raises the bid, the process repeats until one of them accepts or bids 0.5 points. The match is then played for the remaining 1-X points.

Or, more simply, player A is shown N random positions, picks one of them and lets player B pick the color.