Wait, but if those headers are missing, then isn't there a vulnerability if someone is using an old browser and clicks on a malicious link? Do we need to also check user agent or something else?
If you can't afford to do this you still need to use CSRF tokens.