Waterworks H4xx0rd in IL?

[MikeDoyle]

Not a disaster (at this time, at least - might be a better fit under Contingency Planning), but one more failure mode made explicit:

MSNBC: "US investigates cyber attack on Illinois water system"

Now, I readily admit I know next to nothing about IT Security, and even less than that about Utilities Management. IIRC, though, the possibility of cracking against utilities' information systems was in the popular press over a decade ago. Did somebody not get the memo?

On the micro level, what preps are appropriate, beyond preparing for power and water outages we already do for storms and the like?

(Note to Mods - hasty search didn't turn anything about this story, but my search-fu is weak. If' the post is a dupe, I apologize)

[TacAir]

Federal investigators are looking into a report that hackers managed to remotely shut down a utility's water pump in central Illinois last week, in what could be the first known foreign cyber attack on a U.S. industrial system
(Source)

Technical overview - a Master thesis.

And the military? Are they going to protect stuff? Maybe not so much:

"The Air Force made an early grab to be the dominant force in cyberwarfare capability, asserting its authority over the cyberspace domain back in 2005. The Air Force then pushed to set up an 8,000-man strong cyber command to be called the Air Force Cyber Command (AFCYBER).

However, after a shakeup in the top levels of the Air Force in 2008, in which both the Air Force secretary and chief of staff stepped down, the service decided to suspend its efforts to set up the command. An internal Air Force memo dated Aug 11/08 obtained by Nextgov said that “transfers of manpower and resources, including activation and reassignment of units, shall be halted.”

The delay was ostensibly instituted to give the new chief of staff, Gen. Norton Schwartz, time to make a final decision on the scope and mission of the command. But service sources told Nextgov the decision was in response to fierce opposition from both the Army and the Navy, which were both developing expertise in cyber operations.

In fact, the Air Force never did set up the Air Force Cyber Command. Instead, responsibilities for the cyberspace mission were transferred to the 24th Air Force, which was set up in 2009 [PDF] under the Air Force Space Command. Its designation was official changed from Air Forces Strategic to Air Forces Cyber in 2010.

The 24th Air Force is now the service’s component of the US Cyber Command, along with the Army Forces Cyber Command, the Fleet Cyber Command, and the Marine Forces Cyber Command.

In addition, DHS, which has cybersecurity authority over civilian federal government networks, has pushed back on the Pentagon’s efforts to expand its cyberwarfare authority over US critical infrastructure, which is primarily privately owned. "

(Source)
Not ready because of turf wars, who would have guessed?

[ei8htx]

Read the Reuters article. Why exactly do they think it's a foreign attack?

In any case, I don't like that certain politicians wish to increase the oversight of private networks by the DHS or Airforce.

[phil_in_cs]

On the micro level, what preps are appropriate, beyond preparing for power and water outages we already do for storms and the like?

If the water shuts off, the water shuts off. The "why" matters to the people that need to fix it, but not to the people without service.

[JC8]

I think that for most agencies the bigger concern with getting access to SCADA is the issue with water treatment. If water pumps can be turned on and off, then chemical injection pumps can probably be turned on and off. People could be sick before anyone knows that there's a problem (better have Imodium on-hand). Sure, there should be a lot of safeguards built in each SCADA system to help keep untreated water from entering the distribution system, but with the right type of access and enough knowledge it's possible that these safeguards could be overcome. Most SCADA systems are stand-alone and don't connect to the agencies/cities business server, which really helps to keep access buttoned down.

[Spookadelic]

This story came out about the 18th of November, made pretty big news (was on drudge anyways). However I have not heard a single followup story to this. Which means one of two things for me. Its the hack of the century, or it wasn't a hack at all.

I feel there is a good chance, that the whole incident was really nothing, perhaps just an error by an engineer or the SCADA system.

That being said, I used to be a process control engineer (basically works with SCADA systems). I am now a software engineer who specialized in writing software for components of SCADA systems (process data acquisition, process data historians). I also study, practice, and formally trained in ethical hacking (hacking in your own lab, so you can learn how to defeat hackers). I also do this for a very large company that uses LOTS of SCADA.

Ill try to be brief so as to not bore anyone with the heavy technical details.

1. SCADA systems control many many things, if most stopped functioning it would bring a G7 nation to a halt.
2. They are extremely vulnerable, often they have zero security measures.
3. They are hard to keep isolated, especially in a business or municipal type setting.
4. Independent hackers aren't interested very much in SCADA, for them, the money is in stealing your credit card and draining your bank account. SCADA is low on their list, but not off the list, since its a skill that could be used to extort money, and it is a service that could be sold.
5. Governments, and other political entities are VERY interested in SCADA, it will be one of the method they will use to wage "war" (see stuxnet). They are building up their "weapons" (exploits) and they going to sit on them until its time for "war", they are not going to be breaking into small town water systems just to see if they can break a pump.
6. Citizens, Businesses, and Governments should be VERY concerned about this threat, especially those in modern industrial nations.
7. It's one of many reasons I prep.

[JC8]

I've been working for water agencies since 1983, the last 14 yrs. in management, and the SCADA system is part of my responsibility. From a technical standpoint I consider myself an end-user-plus and not a technician.

Prior to 9/11 I'm sure some agencies had the equivalent to zero security measures. After 9/11, and a slew of vulnerability assessments, it seems unlikely that any agency would not have some degree of security. For example, it has become the rule and not the exception to keep SCADA isolated from the business servers. Remote access (if allowed at all) is typically through a VPN and has several layers of password protection. On my system I require strong passwords and they change quarterly (drives the operators crazy). I also get email notification for any failed attempt to log in remotely. Nothing is bulletproof, but at least in the water industry is actively concerned has made concerted effort to tighten up security. I can't really speak to anything other than the water and waste water industry.

Details in the article are few, but it sounds to me like the hacker somehow got his hands on the correct login & password to gain remote access to the SCADA server, and the correct login & password to access the SCADA software. It appears that they may have been taken from an outside integrator (people who set up and program the software). I find that a little disturbing.

[phil_in_cs]

Details in the article are few, but it sounds to me like the hacker somehow got his hands on the correct login & password to gain remote access to the SCADA server, and the correct login & password to access the SCADA software. It appears that they may have been taken from an outside integrator (people who set up and program the software). I find that a little disturbing.

That's a very common method. Much easier to do that than to crack an actually secure system.

[clybourn]

This is the only article I can find on this right now. It mentions a Wahington Post article from the 28th but, I haven't located that.

http://www.gsnmagazine.com/node/25092?c ... tification

[Blood_Moon]

Not exactly OT (skipable):
I recently wrote a paper for my masters program that addressed this issue to some extent. In my research I learned that SCADA is considered secure because it is only used in indistrial applications, meaning the programming language is unique to SCADA and would require purpose built programs written in SCADA to do anything to those systems. I found examples like the famous Aurora Experiment, as well as an incident in Australia IIRC. For those that don't know, the Aurora Experiment was done at the Idaho National Labs where they sent a command to a diesel generator (the BIG kind, like enough to power a small town)that caused it to literally destroy itself. We are talking about chunks of it flying around and black smoke spewing from the exhaust, not something you can just start up after a quick repair. The other incident was not an experiment, but a disgruntled waterworks employee that had been fired that used a wireless connection to back up the sewers, causing significant damage to the surrounding area. However, this was all a few years old I wasn't able to get much CURRENT information on the security, so hoped that maybe they had improved things! Well Spookadelic and JC8 shot that hope all to hell... thanks guys!

Closer to OT (more skipable):
Anyway, the conclusion that I have come to is that the next big attack on the US is going to be one of two main methods. Either it will be a Mumbai/Oslo, Norway style attack with some IED's and gunmen that lay siege to some poor unsuspecting city; or it will be a cyber attack against the infrastructure.

As close to OT as it's gonna get (entirely skipable after first paragraph):
As far as how prepare for the next attack (there will be one... eventually), I suggest that you not limit it to water since there are numerous systems that are vulnerable. Remember, there are really 4 main parts of our infrastructure that could cause direct physical damage, electric, water, gas and roads. (I didn't list communications since that would be more an inconvenience in many cases and would not lead to physical damage except in extreme cases, and may actually help with traffic! ) The electric grid is actually horribly vulnerable since no power company really wants to own the grid due to the cost of maintenance and updates. It is much cheaper to pay to use someone elses grid. The current grid is outdated and were there to be a well planned attack, it could potentially cause a cascading effect that would knock out power for HUGE areas (PM me and I can send you links to the scholarly articles I found on this). This is not a possibility but a known fact, as can be evidenced to by the recent outages that we have seen that were the result of simple tests and procedures that went wrong. The dangers of the water system was already mentioned: contamination, poisoning, etc. I really don't want to know what could be done to the gas system. After seeing footage of that pipe that failed and erupted into a massive inferno, I imagine it wouldn't be good. Besides, "Live Free or Die Hard" covered that already right?
One scenario that I have posted before that may be just as dangerous in the short term involves the other part of our major infrastructure that many don't realize is susceptible to hacking, the roads. In some big cities, traffic lights are linked by a computer system that allows technicians to alter the lights programs remotely in the event of a malfunction. So what would happen if someone wrote a program that simply made every traffic light green, and then implemented it during the morning rush hour? Accidents across the city, some at high speed, all in the space of a minute. You have instant gridlock as all the major intersections are blocked by accidents. You have emergency personnel tied up across the city, emergency rooms inundated with the injured. The roads would undoubtedly take some time to clear since the tow trucks would all be tied up with cars already, and that is assuming that they can actually get through the traffic to the pileups to move the vehicles. It goes on and on... and all because someone decided to give every commuter their greatest wish, that every traffic light on their route to work was green! (obligatory "careful what you wish for" warning ) But you get the idea. Small hack leads to big problems!

[clybourn]

Local news is now reporting it was an employee vacationing in Russia.

[andygates]

Indeed. Not an attack at all: someone saw a Russian IP address and panicked before checking where their staff were.

"If they'd bothered to phone me, all this could have been avoided," said the engineer.

Pure jumping-art-shadows lulz.

[Spookadelic]

I havent looked at this since Nov 30th. Low and behold WIRED had released an analysis that same day.

http://www.wired.com/threatlevel/2011/11/water-pump-hack-mystery-solved/

[TacAir]

I havent looked at this since Nov 30th. Low and behold WIRED had released an analysis that same day.

http://www.wired.com/threatlevel/2011/11/water-pump-hack-mystery-solved/

Thanks for sharing the story and link!

[MikeDoyle]

A) Nice to know - thanks for the follow-up!

B) The potential vulnerability remains. I have to admit that my preps at the moment aren't designed for outages of over 2-3 weeks. Prolly wouldn't hurt to work on that...

[andygates]

A bit of a follow-up on the subject. Some security nerds had a deep look at breaking into common controller kit and it wasn't a pretty picture:

https://threatpost.com/en_us/blogs/look ... ity-012012

"It's a blood bath mostly," said Wightman of Digital Bond. "Many of these devices lack basic security features."

While the results of analysis of the various PLCs varied, the researchers found significant security issues with every system they tested, with some PLCs too brittle and insecure to even tolerate security scans and probing.

There's a parallel in implanted medical devices and this story about a patient with a fancy-pants pacemaker demanding to see the source code: http://www.zdnet.com.au/cyborg-lawyer-d ... 330089.htm - and a similar tale about an implanted insulin pump that connects via unsecured bluetooth.

In all cases, the kit was historically developed in a presumed-benign environment: security was at the gate. Now everything has at least some degree of connectivity, security becomes relevant as much to prevent screw-ups as to prevent bad actors. Some of the controllers in the hack-test crapped out when security-scanned -- and that sort of thing happens day-to-day.