Unfortunately, there are a lot of Al Qaeda type jihadists in Yemen who would very much like to kill U.S. citizens. Otherwise I'd agree with you that we shouldn't be there.
It's a whole chicken and the egg thing.
Taken at face value, US should go and bomb with these drones a whole lot of other people too. And no, they wouldn't be "jihadists". I'm sure there is many many people in the states that very much want to kill other U.S. citizens.