Isn't the main problem to secure the distribution of the key?

> Maybe my site has been hacked and I don’t even know it.

How would the fingerprint help in this case? If the fingerprint is also hosted on the website.