Even without nefarious goals, a drone can simply malfunction and drop on someone's head, which is already dangerous. But if 1 out of 1000 drones is programmed to do something malicious, how would our legal systems address it? Do we even have a system that could prevent something like that? We've never had to deal with autonomous robots roaming among people before.
What I'm most afraid of is the arms race of drones. It will happen before the AI arms race that Hawking and Musk warn about: http://techcrunch.com/2015/07/27/artificially-assured-destru...