Because -O3 enables optimizations that break code with some lesser known cases of UB[1]. So people just don't enable -O3 because they can't be bothered to fix UB in their codebase, or because they think it's a "compiler bug".
There are other reasons for not using -O3 which people already mentioned.
As a matter of fact, Gentoo specifically says in their docs that -O3 breaks some packages[2].
[1] https://stackoverflow.com/questions/57889116/different-evalu...