Of course, you can't do a lottery and then demand the chosen self-identify which party they support, nor can you trust them to be honest. The result is that you can only have people who volunteer.
I think the system we have in Slovenia is quite close to optimal: every party who got at least X % in the last election gets to submit applicants and each polling place gets 6 people, who have to have been referred by different parties, with priority based on the rank of the party in the last election.