Hackers destroy water pump via SCADA abuse | Hacker News Reader

Hackers destroy water pump via SCADA abuse

wired.com

103 pointsmunin40 comments

Good. Malicious crackers, please destroy as many non-safety-critical water pumps as it takes for people to take security on these systems seriously. It seems most of the industrial controls industry is used to operating on a proprietary network, and when moving to IP their guess at security is "uh, firewall?".

This is indeed odd. On my home network, I have a firewall at the edge, a firewall on each machine, and every service requires authentication (cryptographic where possible; username+password over SSL otherwise). It took me about a day to set up, and I'm not even a security person.

It's unacceptable that people whose jobs are to secure computer networks do a worse job than I do for the little computer under my TV.

(Yup, all of my machines at home have a public IP address. Convenient!)

how many users does your home system have? what is your budget like to support them? what is your pain point for "at this overhead we just go out of business?"

it's a lot more complicated than "just do it right".

I'm not saying we shouldn't take effort to do it right, but right now the market doesn't price for security so ...

How many users does a water pump have?

Hell, this probably shouldn't have been networked in the first place.

Many things that should not be remotely controlled are. Control freaks are in control of too much of the world. The next time you are uncomfortably hot or cold in a Walmart, don't bother complaining to the store management, it is all controlled, as are the coolers and freezers, from Arkansas.

That's not even it though? Shouldn't systems like this be air-gapped?

SCADA systems often rely on time-deterministic routing of packets, which TCP/IP doesn't make easy. There are a number of issues with securing them, including the fact that unscheduled downtime can be catastrophic (ergo, no hotpatching without massive work).

There are some good posts on the SCADASEC mailing list. There are some lousy posts as well.

Culturally, SCADA system security is approximately where IT systems were in in 1995 with Windows 95.

Second this, as per other comment I made as well. SCADA systems are generally unstable and very precious about how they are implemented, it can drive people to make a lot of concessions. Thankfully we run enough support staff to ensure physical site visits are possible, which eliminates the security and technical issues associated with trying to tie these things into a WAN.

*edit, inferring that instability can drive people to network these devices for support reasons.

I'm amazed that the concept of "air gap" isn't standard operating procedure at any utility.

so lets say that you don't have the budget to keep a fully trained staff of scada and network engineers on site, 24/7. and then there's a systems failure, and the choice is either "they debug it remotely and fix the problem within an hour" or "the staff drives 2-3 hours to the location where the airgapped systems are and then begins work".

now imagine the system is life-preserving or otherwise critical. those 2-3 hours could be hundreds of lives. wouldn't it be criminally irresponsible to not have the system as resilient as possible?

I would say any system whose failure quickly results in the loss of life should always have properly trained staff either on location or trivially close. Anything less than that would be, as you put it, criminally irresponsible, as is the idea of connecting such infrastructure to publicly accessible networks.

I don't think you'd like what your power bill would look like if your distribution company had to have staff trivially close to all of its infrastructure, 24/7. I work at a utility and all (yes, all) of our stations and substations are unmanned.

To have a crew on site at every station, 24/7, you'd be looking at nearly 1,000 employees at $65-90k, assuming 8-hour shifts.

It's almost like you're saying no one has now or ever built an affordable water supply system that can supply water unless it is run remotely through the internet.

Pretty sure that's not the case since internet controlled systems are solidly in the minority.

That depends on how the cost-benefit analysis came out. You did do a cost-benefit analysis, right?

Personally, I would consider connecting critical infrastructure to the Internet without taking adequate security measures to be the criminally irresponsible thing. Of course, water pumps aren't really "critical infrastructure" in the same sense as, say, nuclear power plants. Hopefully their security is a little more reasonable.

Cost-benefit what?

I work for one of the largest municipal electric utilities in North America. We are a monopoly and do not set our own rates. There is no such thing as "cost-benefit" for us. We spend as much money as we need to get the reliability our regulator demands, and then our rates are set accordingly for us to recover those costs.

It is. Just the loose ones make the papers. We run physical divisions on our control systems, even system to system. Lots of walking area to area to do work but none of it will see each other or any WAN configurations. Thank goodness.

Maybe, but for my parents (i.e. non-tech folks) the solution would be more likely "put more police on the internet to catch those criminals".

Did anyone else notice this choice comment:

“They just figured it’s part of the normal instability of the system,” Weiss told Wired.com. “But it wasn’t until the SCADA system actually turned on and off that they realized something was wrong.”

That's a pretty bad sign that the system is so buggy that at first it seems like a hacking attempt is just "normal" instability.

Anybody that's worked with SCADA systems unfortunately wouldn't bat an eye at this. In my own experience they're horribly issue prone, we've churned through 4-5 different vendors in the last 3 years trying to get past basic stability issues.

What issues did you have?

Unfortunately, writing plc code is like writing in assembly with global access to variables. Unless you test things thoroughly it's really tough to have a system run bug free. Thankfully, most of the processes are pretty simple so its not exceedingly hard to test it.

Stanford EE Computer Systems Colloquium

Control System Cyber Security - State of the State

http://www.stanford.edu/class/ee380/Abstracts/111012.html

"Industrial control systems are used in electric power, water, pipelines, etc. These systems were designed for performance and safety considerations, not security. Traditional IT security technologies, policies, and testing may not apply to these systems. Moreover, there is currently no university with an interdisciplinary program accross multiple engineering disciplines to address control system cyber security. There have already been more than 200 actual control system cyber incidents to date, though most have not been identified as cyber. In the US alone, there have been 4 control system cyber incidents that have killed people, 3 major cyber-related electric outages, 2 nuclear plants shut down from full power, etc. With the advent of Stuxnet, cyber has been introduced as an offensive weapon. The purpose of this presentation is to provide a state-of-the-state view of control system cyber security."

The speaker gets quite a grilling from the academics.

Do you know of any videos of the talk itself? That page only seems to list the abstract and a very handwavy CNN report clip.

From the abstract/slides, it seems like it could be quite interesting.

So they stole username/passwords the SCADA vendor kept for the clients.

There are better ways to gain access to systems. But even then, you have to prevent access to your own systems first...

I wonder if the glitches were that they remote people couldn't login. So they kept changing the passwords, and restoring them, and then the hackers just went and got the password again, changed it...

So let's take bets. Was this hack attack made possible through:

1) A SQL injection

2) A default admin password

I'd go for unpatched VxWorks system.

The article mentions that the intrusions involved a security hole in PHPMyAdmin, so, likely neither (unless the attackers just got access to the database through PHPMyAdmin using the default admin password rather than a security hole).

Why isn't using PHP for security sensitive work considered illegal yet?

PHPMyAdmin is just some software used by clueless neophytes, not part of PHP. Similarly poorly written and insecure software is surely possible with Ruby, Python, you name it.

3) USB keys dropped in a parking lot.

They aren't even talking about what happened in San Diego :-\

SCADA controls those power systems too ;-)

So the Blackout was due to a SCADA hack?

The system is redundant, and no single operator should be able to do what was done that day. The military bases went delta-5. Largest military port city, and China's looking to let the fire out of their dragon...

Dark days, eh?

Never thought about it that way, but you are right, a system so vital and critical should never be left for 1 human to deal with, reminds me of missile systems where you need 2 keys and Passphrase's, did they ever resolved this? I'm guessing no.

This seems like an exciting decade we are about to enter where hackers can mess with actual physical infrastructure. Sooner or later somebody is going to do something really destructive with that power.

Fortunately it shouldn't be that hard to secure the systems. At the very least use a two factor authentication system, if possible the same way gmail does since it is pretty simple, or just store the passwords in a big physical folder and access as necessary.

Exactly what good does "two factor authentication" do when every verb in the protocol was designed with the assumption that the protocol would only ever be addressed with an authorized client? These things are insecure by design, insecure in implementation, and insecure at deployment. Don't trivialize the problem; it's immense.

Maybe password theft was involved this time, but that's a trivial detail. I don't feel like endorsing feel-good measures. A lot of this code really needs to be forklifted out, which is a fact made especially painful because a lot of this code is already pushing the limits of the 8 bit TI microcontrollers it runs on.

Is the controller really the place for putting in security measures? I would have assumed that they reside on a private internal network and it's the software layer on top ( ie, a bacnet gateway or whatever) that needs to be fixed

Last time I saw a system like this, it had two distinct networks, one for the controllers and one for the computers that controlled them. I would imagine nobody would allow any direct external access to the controller network.

Sadly, that decade was two years ago. It's been well known how broken SCADA systems are, and shouted from the rooftops, but no one cares. We have been on borrowed time now for ages. It finally happened.

The Shelbyville branch of Anonymous strikes again. Hoot hoot!

Reply on news.ycombinator.com