No, F-Droid builds almost all apps from source. Even some open source apps don't make it to F-Droid if the F-Droid maintainer doesn't manage to build it themselves on their build server.
Additionally, F-Droid signs every app themselves.
Additionally, F-Droid signs every app themselves.
edit: When the signing is in single entity like f-droid, we have single point of failure.
When everybody sign their own app, we have trust scalability issues -- "trust" just can't scale to everybody.
Of course the build platform being compromised is possible but that can happen even with binary distribution.