This happens in many ways with potentially catastrophic tech. There are many formal agreements and strong norms against building ever more lethal nuclear arsenals or existentially dangerous gain of function research. The current system is far from perfect, the world could literally be destroyed today based on the actions of a handful of people, but it's the best we have come up with so far.

If we as a society keep developing potential existential threats to ourselves without mitigating them then we are destined for disaster eventually.