The VC-backed pressure will ensure that all kinds of safeguards will be skipped, regardless of my feelings! Solving millenium math problems in a few hours might piss off some mathematicians. But an AI that's autonomously looking for cold fusion or a cure against cancers might cause planetary-scale desaster on the way there.

The only reasonable safety measure is to always keep humans in the loop, and to give them time to understand and consolidate all advances, to actually go slow.