In both cases, you trust the publisher and in both cases the publisher gets equal access to your machine.
Oh - you mean you're downloading the source code, then audit it, then compile it and only then you run it?
That's super great. That has saved you from the xz backdoor and all other supply chain attacks and will be of great help to you in the future. Let's hope no backdoor ever slips past your code review.