Here's a material example: OpenBSD has expended considerable effort into removing ROP gadgets from their compilation products[1].
As the website documents[2], these efforts haven't made a single exploit harder to write.
GCC even removed their (mostly equivalent) ROP mitigation approach, documenting it as "fairly ineffective" and "lur[ing] developers to the land of false security"[3].
(Another good example is PID randomization: OpenBSD added this over 20 years ago as part of their "randomize anything that can be randomized" approach. It's never had any real positive security impact, and has made other PID-based vulnerabilities more viable.)
[1]: https://www.openbsd.org/papers/eurobsdcon2018-rop.pdf
[2]: https://isopenbsdsecu.re/mitigations/rop_removal/
[3]: https://patchwork.ozlabs.org/project/gcc/patch/CAFULd4ZL-wa3...