If the vote with that number was actually cast, that is the vote that is actually used. If not, try the next one until one actually was cast. Pretty repeatable, and recounting is easier.
[1] Note that "all candidates" may not be sufficient, since it allows the candidates to collectively choose the outcome by picking their numbers such that one of their own/friends' ballots is chosen. This could be used to elect a popular Labour politician in a remote district where only the major two parties are running, in exchange for electing two unknown Tories.
[2] If you're not into crypto, put tickets containing every possible ballot number in a big barrel. Have a trusted/blindfolded guy pick tickets until he finds one that corresponds to a vote that was actually cast. There are lots of solutions, of varying degrees of cleverness.
[EDITs: added two footnotes]