Even then, that only gives you a stronger indication that the image hasn't been altered since it was signed by the image author at any point after it being signed. However it is not a guarantee that the source produced the binary content. It's also not a guarantee that the image author knew what they were signing - though this is a different issue.
Debian has a reproducible builds initiative[1] so people can compile packages themselves and them match byte for byte what Debian built. Not sure how far they've got with that.