You call it a sick way to mess with people, governments will refer to it as cyberwarfare. I'd say that the US / Israel's attacks on the Iranian uranium enrichment facilities (Stuxnet) was just a tame and reserved trial, and that most modern countries have plans in place that will unleash a devastating cyberattack on loads of internet-connected devices.
I'd say routers are particularly vulnerable, if only because they are smart (Linux) machines, but in most cases users will never check them for anything odd going on. As this article shows, it takes but a simple command for them to execute stuff, and given how Linux is a general purpose OS, they could install and perform any kind of task - like install backdoors and whatnot on the PC's behind the routers, which can then in turn be disabled or used in a massive botnet to perform a DDoS or other attacks on other systems.
Just think about the implications of there being a backdoor in every internet-connected computer system, or the consequences of all-out cyberwar.