At least we can turn it off... for now. Does anyone know what happens to your unsigned apps when you upgrade from a pre-ML OS X to ML? Do the programs that used to work suddenly fail? If so, that is a terrible UX!
It's basically identical to the "You just downloaded this file from the web, are you sure you want to run it?" check built into Safari - the file quarantine system http://support.apple.com/kb/HT3662?viewlocale=en_US&loca.... However, this defaults to "No" for unsigned code, and "Yes" for signed code.