Encryption/signing is only for proving that the source is who it claims it is. It doesn't mean that you can trust the software itself. Towards that end you've got source-code. Ubuntu's software packages are built from source and you can inspect it - yes it may not be feasible, leaks may escape even trained eyes and so on and so forth, but the source-code is there available for inspection and nothing short of a source-code review can prove that the software does what it claims.