A backdoor/bugdoor is much more likely to be in the source code. Compiling the code by yourself does not help.

You have to rely on a distribution that has a many-eyes review policy and has security conscious users.