Some drones are operated by a division of the CIA and are out of military control, however. This is probably the most worrisome aspect of the program.
However, as long as 99% of their targets are bad guys, Americans are going to continue supporting the program. It's zero risk for American pilots, and everyone likes to read about more dead terrorists with their morning coffee. More points to whoever's in charge.
Governments like Yemen's will also support it because #1 it does their dirty work for them, and #2 they can blame an evil foreign power. Win-win.