Potentially unsafe optimizations, undefined behavior and random crashes that are hard to reproduce are normally nothing one wants to have in production builds.
It's nice that UBSan will detect the problem at runtime, but that requires one to actually use UBSan in development and have all relevant code paths triggered in a UBSan-instrumented build. Which is unrealistic at least when 3rd party code (libraries etc.) comes into play that is not entirely under the application's control. So I would really favor optimizations that do not make a program less safe, or are only applied when can be proven that they do not cause any harm.